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ABSTRACT 


An adaptive recursive digital filter is presented in 
which feedback and feedforward gains are adjusted adaptively 
to minimize a least square performance function on a sliding win- 
dow averaging process. A two-dimensional version of the adap- 
tive filter is developed and its performance compared with 
the optimal Wiener filter. The filter is shown to be effec- 
tive in Separating three diagonal trajectory streaks from a 
background of correlated noise added to white noise. Although 
the recursive adaptive filter approaches the optimal Wiener 
filter in performance, it does not require a priori statistical 
knowledge as does the Wiener filter to which it is compared. 
The results indicate that the recursive adaptive filter "learns" 


the statistics and adapts. 





int. 


ELI. 


TABLE OF CONTENTS 


Teh ie Oe) re e. nen ass - 2-22: 7 
ARE SCH ASBIESEROCESSES -- 222 2.2.2.2... ----2- -22- 10 
PIE Romanen E a 00 10 
B. DEFINITION AND PROPERTIES OF LINEAR 
See USEOCESSESE en rennen nn i 
C. MODELING OF LINEAR STOCHASTIC PROCESS ------------ 22 
l. Introduction -------------------------------- 22 
2. Method of Least Squares --------------------- oS 
3. Method of Filter Response ------------------- 34 
A TR Ecc zT 43 
DENM ISMINESUPESTVESETPEITERO S --------2--2.---2-2-.---.-.-2--— 43 
l. Introduction -------------------------------- 43 
2: Performance Surface, Gradient and 
Wiener Solution------------------------------ 45 
3. LMS Algorithm for One-Dimensional Filter ----- 48 
4. Two-Dimensional Adaptive Filter ------------- 52 
a. Adaptive Filter Structure --------------- 52 
b. Wiener Solution ------------------------- 54 
SA SAO ENM A Seesen = = 56 
E E SU A E Se SS Se 58 
IP SS MEE Ole = a = Re nae 58 





2. One-Dimensional Adaptive Recursive 


Filter ------------------_----------2------- 59 
A. Dee ee 59 
E O EN Tener LUCIO e 61 
€. AO A o e a ias 64 


3. Two-Dimensional Recursive Adaptive Filter 


E D LIRBENIOERSESCANGEDLER sas a oi > jaa SS p 
A. THE CONCEPT OF ADAPTIVE NOISE CANCELLING ------- 77 

B. NOISE CANCELLING WITHOUT AN EXTERNAL 
E en en 82 
DEREN LAND RESULTS 772 200220220 econo 85 
EE Gg en e ecco cmRRIII-I2.- 105 
Beer n 108 
BEE EDTERTLBUTTON BIST -- 2... .2.--.2..2-- -------222- 109 





EN TRODUCTTON 


The term "filter" is often applied to any device or 
system that processes incoming signals or other data in such a 
way as to eliminate noise, to smooth signals, to identify each 
Signal as belonging to a particular class, or to predict the 
input signal from instant to instant. There is an abundance 
of literature covering the theories involved under the head- 
ings of estimation, identification, modeling, prediction, etc. 
The usual method of estimating a signal corrupted by noise is 
to pass it through a filter that tends to suppress the noise 
while leaving the signal relatively unchanged. The design of 
“ such filters falls in the domain of optimal filtering, which 
originated with the pioneering work of Wiener [8] and was ex- 
tended and enhanced by the work of Kalman-Bucy [9] and others. 

Filters used for the above purpose can be fixed or adap- 
tive. The designof afixed filter is based on a priori know- 
ledge of both signal and noise statistics. On the other hand, 
adaptive filters have the ability to adjust their own para- 
meters automatically, and their design requires little or 
no a priori knowledge of signal or noise characteristics. 

This work presents an approach to signal filtering using an 
adaptive filter that is in some sense self-designing. 

The adaptive filter described here bases its own "design" (its 
adjustment of internal parameters) upon estimated (measured) sta- 


tistical characteristics of input and output signals. 





The statistics are not measured explicitly and then used 
to design the filter; rather, the filter design is accomplished 
in a single process by a recursive algorithm that automatically 
updates the system parameters with the arrival of each new data 
sample. It is assumed that the input and the output at the 
sampling instants are the only measurable quantities of the 
system. It is also assumed that the unknown filter coefficients 
parameters) to be designed enter linearly in the difference 
equations which describes the self-designing process. 

The steepest descent method is employed in which the pre- 
vailing filter parameter vectors are perturbed at each itera- 
tion in a manner so as to decrease a prescribed functional 
(error en or cost function) to be minimized. The steep- 
EL descent method is one of the well known gradient based 
algorithms. 

For the case where the functional being minimized is the 
mean square error, where the error is the difference between 
filter output signal and the desired signal, the filter is 
called the least mean square filter (LMS filter). Various 
adaptive algorithms are currently available depending upon 
the cost function and the method used to minimize cost function. 

The popularly used performance criteria are the least mean 
square criterion, the maximum likelihood ratio (MLR) criterion, 
and the maximum signal to noise ratio (SNR) criterion. Here 


the LMS criterion only is studied and the steepest descent 





method is employed. Inevitable errors in the estimation of 

the statistics prevent the adaptive filter from delivering 
optimal performance since the adaptive filter is not based on 
the a priori knowledge of statistics. In Chapter II, the con- 
cept Of linear stochastic processes is reviewed as a preliminary 
study for this thesis, and the modeling of stochastic processes 
is studied. These can be considered as background material 

for the following chapters. 

In Chapter III, the concept of adaptive filters is intro- 
duced and the structure of the signal and the mathematical 
model of the processor is delivered. The algorithm for the non- 
recursive adaptive filter by Widrow [1] is reviewed and the new 
algorithm for the recursive adaptive filter is developed as is 
the two-dimensional adaptive filtering process. 

In Chapter IV, the adaptive noise cancelling concept is 
analyzed rather qualitatively and its application to the special 
case in which no desired signal is available is analyzed. [In 
Chapter V an experiment is performed through computer simulation 
to check the feasibility of algorithms developed in the pre- 
vious chapter and a comparison with the optimal Wiener solution 
is made. In Chapter VI, the conclusions are presented together 
with a summary of the experimental results and suggestions for 


further research. 





NERO TOCHAASTIE PROCESSES 


A. INTRODUCTION 

The problem of defining a random process is of considerable 
importance in the analysis of systems subject to noise distur- 
bance. Often a partial definition of the process will suffice 
Ene casce of linear least mean square error filtering, 
where only a knowledge of the correlation function is required. 
For other problems, such as those involving nonlinear filtering, 
more complete information will generally be needed. A complete 
description of a random process requires a knowledge of the 
distribution functions of all orders. But in practice few 
processes apart from the normal and Markov are defined in this 
manner. For the purpose of analysis, a model to generate the 
random process is desirable and for a model to give a complete 
description of the process, the distribution functions should be 
derivable from the model. While both continuous and discrete- 
time linear process may be defined, only the discrete-time case 
will be considered here. The discrete-time linear process 
can be considered to be the result of the digital filtering of 
a sequence of independent and identically distributed (IID) 
random variables. 

The linear processes are important since they are inherently 
simple in terms of physical considerations and form a class 
which includes many discrete time normal random processes. 

In the following section the definition and properties of the 


linear processes are summarized. 





ERZERELTNETTON AND PROPERTIES OF LINEAR STOCHASTIC PROCESSES 

It has been found useful in the theory of stochastic processes 
to divide stochastic processes into two broad classes: sta- 
tionary and nonstationary.  Intuitively,a stationary process 
is one whose distribution remains the same as time progresses, 
because the random mechanism producing the process is not chang- 
ing as time progresses. A nonstätionary process is one which 
is not stationary. 

Rete er) be a stochastic process with finite second 
moments. Its mean value sequence , denoted by m(i), is defined 
mor all i in T by 

Er} (Zi) 
and its covariance kernel, denoted by K (j.i), is defined 
or all j and i in T by 

MESE = Cov x (7), * (1)] 22) 
An index set T is said to be a linear index set if it has the 
property that the sum itj of any two numbers i and j of T also 
belongs to T. Examples of such index sets are T = (1,2 , . . .]), 
D rn. .}, Ieli;i2o} and T=eli:— <i<o} . 
A stochastic process {x(i), ieT ), whose index set T is linear, 
is said to be 

1) strictly stationary of order k, wherek isa given positive 
murder, it any k points i, 1 + 1, ...i +k in Tt where 
t+ A {x (i), i>o}, and any j in Tt, the k dimensional random 
vectors {x(i), x (itl). . x (itk)}and{ x(itj),. . . x(it+j+k)} 


are identically distributed. 





ii) strictly stationary if for any integer k, it is 
strictly stationary of order k. 

iii) widesense stationary (covariance stationary) if it 
possesses finite second moments, if its index set T is linear, 
and if its covariance kernel K (j,i) is a function only of the 
absolute difference |j-i|, in the sense that there exists a 
function R(n) such that for all j and iin T* 

K (j,i) - Covix(3), x(i)] » R,, (3-1) (2-3) 


or more precisely, R.,(m)has the property for every i and j in gu 


ovo Tx(2) x (m) ] EB eta) x (1m) | 
- Ryx (m) (2-4) 

We call R,.. (m) the covariance function (autocorrelation function) 
Of wide sense stationary time series {x(i), ieTt} . 

The second problem concerns the concepts of ergodicity 
and the strong law of large numbers in terms of linear processes. 
TO present a complete discussion of this question is not rea- 
sonable for review purposes, but it is interesting to con- 
sider certain aspects of it. For strict sense stationary pro- 
cesses, the ergodic theorem is the strong law of large numbers 
and states that 

if(x(i),ieT!) is a strict sense stationary, ergodic 

random process and E[|x(o) |]<«, 


m 
then lim " 2 x(i)=E[x(o)]with probability 1. (2-5) 
m i=l 


eZ 





In general, a stochastic process is said to be ergodic if it 

has the property that sample (or time) averages formed from 

an observed sequence of the process may be used as an approxima- 
tion to the corresponding ensemble average. 

Stationarity and ergodicity concepts are readily extended to 
two-dimensional random fields (for two dimensional signal, the 
term random field is preferred to random process). 

The assumption that the field is stationary means that 
the statistics of a point in the fieldare not dependent on the 
location of the point. Then, a stationary, two-dimensional 


field has an autocorrelation function defined as: 
Ry (M,N) A Elx(k,2)x (k+m, (+n) ) (2-6) 


and it is also said to be ergodic if the statistical (ensemble) 
average of random field x(k,l) is equal to the spatial averag- 
meg Of all points. That means 

E[x(k,A)] = <x> (22) 


where «x» by definition represents spatial averaging 


E lim l TL MiM: 
OOT Wo E LIT x (k,1) (2-8) 
2 90 i 20 O 


Now, consider a stationary sequence of random variables 
{x(i), ieT}. The correlation function of the sequence may 
be written in the form 

Rx (nk e ^"""  gF(w) (o) 


IT 
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where F(w) is a nondecreasing function, called the spectral 
distribution. If F(w) is absolutely continuous with F (w) = 


f(w) almost everywhere, we may write 


nm  iwn 
Rzy (n) = e f(w) dw (2-10) 
-T 
Under certain conditions, (e.g. E |R(M)|<w, finite), the 
n=0 — 


correlation function may be inverted to yield the spectral 


density as 


f(wm= + rz R,,(n)e iP" (2-11) 
2T -o 


A sequence of random variablesíx(i), ieT) is said to be a 
process of moving average if it admits the mean square repre- 


sentation 


x (i) = h(i-3) u(j) (2-12) 


whereíu (3), jeT) is a collection of orthornomal random 
variables (sequence of white noise). If the sequence {h(i), 
Ns tone Sided (ie, h(i) = 0, 10), then {x(i), ieT) is 
called a one sided moving average process. 

A process is said to be regular if the error for prediction 
one time unit ahead is nonzero. It is known that a process is 
one of moving averages if and only if its spectral distribution 
function is absolutely continuous. Furthermore, a process with 
an absolutely continuous spectral distribution function is 
regular if its moving average representation is one sided. These 


facts serve to motivate the following definition of a linear 


process. 
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DEFINITION [5] 
A linear process {x(i) ,‚ieT} is one having the structure 


E j 
SS eh (jai) u(t), (2583) 


O ax OO 


where {u(i), ieT} is a sequence of independent and identically 
distributed (IID) random variables. The set of real constants 


Baron, ieceTt} is such that X[h(ài)| «e, and the function 
(e 
MAA where z is a complex variable, is analytic 
O 
and has no poles outside the unit circle in thez plane. The 


correlation function of this process is given by 


Ría) = F h(3)h(j*n) (2-14) 
and the corresponding spectral density f(w) is 
E Law ee i 1W||2 
f(w) = — |% h(j)e"2*/? =-— | H(e**) | (2-15) 
2 O 27 
It is assumed further that the process (2-13) has a rational 


spectral density, that is 


B (eiV) 


- A (2-16) 
A(e!W) 


£(w) = 1 | n(el) |? - 
T p 








where both Ale Y) and B (eV) are polynomials in ei of 

finite order with all their poles inside the unit circle. 

The process (2-13) may be generated by passing the sequence 
{u(i), ieT} through the digital filter H(z). By the assumption 
in the definition of the linear process and by the restrictions 


on the spectrum, there exists an inversion D(z) where 


D(z) = A(z) = È} 


B(2) H(z) 


Mm general, D(z) and H(z) may be infinite polynomia in 2. 


D(z) may be written as the one-sided sequence 


E -i E 
D(z) 2 diz (2-17) 
Passing the x(i) sequences through the digital filter will 


recover the generating sequence u(i), that is 


j ES 
u(j) » E dj-ixi)9 5 dü)xg-i) (2-18) 


00 


This is called an operation inversion. In general, we will have, 


assuming ay = 1 
" E: 
2D ,Z = co ES 
H(z) = II E m = EX h(i)z (2-19) 
1 + a.z O 


ù l 


and the process{x(i), leT) may be represented in two ways 


i) x(3) = jp h(iu(-i) 
(2-20) 
n 
ii) x(j) = z biu(j-i)- Paix (5-4) 


The second representation indicates that if H(z) is an all- 


pole function, then we have 


21 ) z a x(J=71) = u(j). (2-21) 
In this case, since inversion uses only a finite number of past 
samples, the process is called "finitely invertable." It is 
clear that finitely invertäble linear processes form a subclass 
of autoregressive schemes for which case the set {u(i)} in 
(2-21) would be orthonomal rather than independent. The con- 


cepts and definitions above can be readily extended to a 





two-dimensional linear process. A two-dimensional linear 


process {x(m,n), Mez, nez>) could have the structure 


x(m,n) » ZEZh(k,f)u(k-m,Ü-n) 
00 


mn 
= È h(k-m,!-n)u(k, 9) r (2=22) 


where fu(k,2), kez ,, lez} is a two-dimensional sequence of 
IID random variables with zero mean and unit variance. The set 
of real constant {h(k.2), kez", ezt} is such that 


k, | 


ÈL | h(k, £) |<< "and the function H(Z1,225) = aae > ! 


where Z4 and Z, are complex variables, is analytic. The correla- 


tion function of this process is given by 
R (myn) = Zih(k,AJh(k+m,k+n) (2-23) 
00 
and the corresponding spectral density £ (w4 ,W») is 
E 1 2 
uou) = (2) 


2 : - 
2 (H) H(olVl, Qiw2) (2-24) 





= h(k,L)etwık emet 
00 


with the same reasoning as in the one-dimensional case, we have 


Be general, 


= 2 E | 

es Pee az. 2 -J 

O O ij) l 2 

k > 
Tan? aij Z 


EIC 7 


qe . 
= 
l 2 
i=0 j=0 
(i1,3)%(0,0) 


T co 00 -k = e 
- » h(x,2)2, Za e (2-25) 





and the process {X (k,l), keZ), lez,}may be represented in two 


ways 


i) x(k,@) = ¿2n(i.3)u(k-i, 1-3) 


En N a u Lie E 
mm) x(k,) = z 13"! -1,57J) - D» 2 aj¿X(k-i,-3) (2-26) 


(1,3)4(0,0) 
It should be noted that the moving average scheme would be ac- 
complished by passing the orthonomal IID random variables through 
the nonrecursive filter and the autoregressive schemes through 
the recursive filter for both one-dimensional and two-dimensional 
random process. 

In the study of systems subject to the random signals, the 
concept of power spectral density function is of importance. 

For a given transfer function of alinear filter, the cross power 
spectrum between input and output of the filter, and the output 
power spectrum is of primary concern. 

Consider a (continuous) system subjected to a random input 
signal. Given a linear system with a transfer function 
Ew). the input to the filter a stationary process x/ít) 
with an autocorrelation function po (iz) and a power spectral 
density function GW), then the following relationships 
are obtained 

Gzy (W) = G y (W) H* (jw) 

Gy v) = Gy (w) H (jw) (2-27) 

Combining above two equations, 


: = 2 = 
Salz Gy (w) {H(jw) |“, (2-28) 
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wh ere A is the cross power spectrur and Gy (Ww) 1s the 
output power spectrum, the cross correlation would be cal- 
culated by using the inverse Fourier transform of EIE 
For the continuous two-dimensional linear system of 
H(Jw, +JW)), subjected to a stationary two-dimensional input 


signal of power spectrum Gy (W1? Wo) - 

Sy (71 7 w2) = Goy (Wy 1 W5)H(JW4 ,JW2) 

Gyy (W7 1 Wo) = Gey Wye wa) E Jwi? JWo) (2-29) 
Again combining above two relationships, the output power 
spectral density function is 

" ! | 2 B 

Goa (Wy Wo) = Gy (wy, Wo) |H(jwy, jw.) | (2-30) 
Consider a discrete linear system to which a random input se- 
quence is applied with a transfer function H(z). The input 
sequence is a stationary process x(1) with an autocorrelation 
Bunction Ry (m) and its 2-transform Gzy (Z)? where Gy (Z) is equi- 


valent to the power spectral density in the continuous case,i.e. 
A 


Guy (2) - IR, (m) ] (22:315 
Then, 
E = zi) H(z) 
M -1 
Gy (2) = Gex (Z) H(z)H(z `) (2-32) 


For the two-dimensional discrete case, 


Gyy (211 Zo) 5 or za Ze 25) 


zu 


ET -1 = 
and G - = G x C21! Z5)H(2Z,,25)H(2, 1 Zo ) (2-33) 


IS. 





As a summary of this section,a discrete-timelinearprocess can be 
Bensidered to be the result of digital filteringan indepen- 
dent identical random sequence having zeromean and unit variance. 
The moving average scheme is the result of filtering through the 
nonrecursive filter and the autoregressive scheme is that of 
filtering through the recursive filter. 

And for a linear system, the relations between transfer 


function, power spectrum, and auto correlation are given by 


Ea Es |H] 


O xy = Sys = 


R (autocorrelation function) = 3 transform of power 


spectral density function. 


The Figure 2-1 shows the block diagram which describes 


the various relations and concepts. 
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C. MODELING OF LINEAR STOCHASTIC PROCESS 


IAE Ecoductlon 
A more active concern at this time is that of system 

modeling. It has been shown in a previous section that a linear 
stochastic process (or field) can be generated by filtering | 
white noise through a linear filter. The problem can be stated 
as follows. What is the filter equation (difference equation) 
that produces a typical random process with a specified autocorrelation 
function? That is, with the knowledge of second-order statistics, 
determine the filter coefficient a's and b's in equations (2-20) 
and (2-26). It is clear that if one is successful in developing 
a parametric model for the behavior of some random process, 
then the model can be used for different applications, such as 
prediction, estimation, smoothing, etc.. As far as the general 
modeling problem goes, one of the most powerful models currently 
in use is that where a signal x(n) is considered to be the out- 
put of some system (filter) with unknown input u(n) such that 
the following relationship holds 

P 


x(n) = - X 


q 
: ayx(n-k) * GE, bgu(n-1), b,-1 (2-34) 


1 
where ayı l< ks p , b2 1 <%<q 

andthe gain G are the parameters of the hypothesized system. 
NemMation (2-23) says that the "output" x(n) is a linear function 
of past outputs and present and past inputs. That is, the 


signal x(n) is predictable from linear combinations of past 
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Outputs and inputs. For the two-dimensional case, the difference 


equation corresponding to equation (2-34) would be 


M, M, i a iy 
X(k,2) = - Y Lo aQ.Qx(k-i,£$-j)tG 2 2 bol 90 (k=2 7 $715) 
i=0 j=0 J £.-u £,5-u i 
l 2 
where (i,j)7(0,0) mn) 


Equations (2-34)md(2-35) can also be specified in the frequency 


domain by taking the z transform of both sides of Eq (2-34)and Eq 





(2-35). 
=2 
MEX (ZO. i+ gui hs 4 (2-36) 
H(z) = — GG —————————— 
EN p E- 
l+ X a,2 
k=1 
where X(Z) = EX x(n)g? is the Z transform of x(n), and uU(Z2) 
n= c) 


is the z transform of u(n). 
For the two-dimensional case, 


L L mr Zr 


l 2 m 2 
b Z 7 (2-37) 
» 2 Ly La 1 2 
X(27 122) 2,=02,=0 
172 M, M ; 
D Z7.) 1272 -i -j 
1772 l 4 OS 2,427 2, 
i=0 3j=0 J 
(i.j)# (0.0) 
where X(2],2,) E 2 [x(k,2)] 
U(Z,,Z5) = Slu(k, 2) ] 


H(Z)andH(Z,,25) inEgs (2-36)ad(2-37) are the general pole zero 


models, 


2S 


The roots of the numerator and denominator polynomials are the 
zeros and poles of the model, respectively. There are two 
special cases of the model that are of interest. 
i) all- zero model Ak = 0, Aij=0 
ii) all- pole model b,= 0, blita = 0 
1<2% <q 


0 


|^ 


k< L) 


0 


[A 


£as Lo 
But (24, £5)7 (0,0) 

As mentioned in section II-R, the all-zero model is known 
in the statistical literature as the moving average (MA) model 
and the all-pole model is then known as the autoregressive (AR) 
model. The pole-zero model is then known as the autoregressive 
moving average (ARMA) model. It should be recalled here that we 
are interested in the case where u(n) or u(k,!) is white noise, and 
this will be treated as a special case in the following. The 
modeling problem can be stated as"given signal x(n) or x(k,Q), 
find the filter coefficients (a's and b's)and the gain, G,in 
Equation (2-34) in some manner." Two approaches will be given for 
a solution of the above problem. The first isttemethod of least 
squareswhich is based on the optimal estimation concept, and the 
second isthefilter response method in which linear system pro- 
perties are used. The one-dimensional case will be treated 
first, then two-dimensional case including some examples. For 


example, a lowpass correlated random process (field) and band 


24 


limited random process (field) are chosen since they represent 


practical examples. 


2. The Method of Least Squares 
Athoughthefollowing can be applied to the deterministic 


Signal and stochastic processes, stationary or nonstationary, 
it is emphasized for only a stationary random process and only the 
all-pole model is considered [6]. In the all-pole model, we 
assume that the signal x(n) is given by as a linear combination 


of past values and some input u(n): 


x(n) = = 


I m 


A,x(n-k) + Gu(n) (2-38) 
k 


T 

where G is a gain factor. 
Here, it is assumed that the input u(n) is totally unknown, 
which is the case in many applications. Therefore, the 
signal x(n) can be predicted only approximately from a linearly 
weighted summation of past samples. 

Let this approximation of x(n) be Sin), where 


a = - = Ayx (n-k) : (2-39) 


k=1 


then the error between the actual value x(n) and the predicted 


value a is given by 


P 
e(n) = x(n) - ND = x(n) + 2 Ay Xx (n=k) . (2-40) 
k=1 
e(n) is also known as the residual. In the method of least 


squares, the parameter Ay 'S are obtained as a result of the 
minimization of the mean square error with respect to all of 


the parameters. 
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If the signal x(n) is assumed to be a sample of a random 
process, then the error in equation (2-40) is also a sample 
of a random process. In the least square method, we minimize 


the expected value of the square of the error. 
2 i 2 
E[e^(n)] = El[x(n) + 2 A,x(n-k)]} (2-41) 
k=1 


Ele“ (nm) 1 is minimized by setting 


lem 
3A. 


L 


A (2-42) 


From (2-41) and (2-42) we obtain the set of equations 


E 
o Àk Bfx(n-k)x(n-i)] = -Elx(n)x(n-1)] 12iíp (2-43) 
k=1 
Then the minimum average error is given by 
E, = Elx“(n)] + É AkE(x(n)x(n-k) ) (2-44) 
k=1 


For a stationary process x(n), we have 

E[x(n-k)x(n-i)] = Dee) 
Pere R (1) is the autocorrelation of the process. 
Note that equatinns (2-42) and (2-44) lead to the well known 
orthogonality principle [7]. Since in the least squares 
method, we assumed that the input is unknown, it doesn't make 
much sense to determine a value for the gain G. However, 


there are certain interesting points that can be made. 
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Equation (2-39) can be written as 


x(n) = - A, x (n-k) + e(n) (2-45) 


IM tg 


k=1 


Comparing (2-45) and (2-38), it is seen that the only input 
signal u(n) that will result in the signal x(n) as output is 
that where Gu(n) = e(n). (2-46) 
That is, the input signal is proportional to the error signal. 
e(n) can be also considered as the modeling error. The error 
variance can be calculated by Equation (2-41) and the filter 
coefficient A, (k=l . . . p) would be calculated by equation 
(2-43) if the correlation function of process x(n) is available. 

At this moment, it should be recalled that a linear random 
process is generated by linear filtering of whitemise. Therefore, 
we are interested in white noise inputs for the purpose of 
modeling a given linear random process. That is, the 
input u(n) is assumed to be a sequence of uncorrelated samples 
(white noise) with zero mean and unit variance. 

Elu(n)] = 0, E[u(n)u(m)] = ómm 


Then the output x(n) forms a stationary random process 


x(n) »- $ Ayx(n-k) * Gu(n) (2-47) 
k=1 
Multiplying equation (2-34) by x(n-i) and takirg the expectation, 
Elx(n)x(n=3)] =- $  AyE[x(n-X) x (n-i)] 
TE [Gu (n) x(n-i)] (2-48) 


Noting that u(n) and x(n-i) are uncorrelated for i>0 and re- 


calling that for stationary process, E[x(n)x(n-i)]-R,, (i), 








equation (2-35) turns out to be 


I m 


Rex (Í) 2 


e A, RL, (1-k) for peo (2-49) 


1 
and R(o) can be obtained by plugging x(n) of Equation 


(2-38)into Equation (2-48) 


E 2 
Ber (O) = =>- py k) +G (2-50) 


AR 
Det 


Therefore, the gain can be given by 


2 _ p 
tance o R. (k) Dee 


Ak XX 


k=1 
It is noted that through Equatinn (2-46), that is Gu(n)=e(n), 
the white noise input of zero mean and unit variance generates 
the random process e(n), which is again white with zero mean 
and variance of c^. Therefore, from Equation (2-49), the re- 
cursive filter coefficients Ay, (k=1, lp can be-calculated 
and using these values the gain G would be calculated by Equation 
(2-51) with the knowledge of autocorrelation function of a given 
class of linear random process. So far, modeling of one- 
dimensional stochastic process has been considered. Similar 
reasoning can readily be extended to the modeling : of two- 
dimensional random fields. Again, the two-dimensional all-pole 
model is considered. 

Mı M2 
X(k,£)--r > Aj¿X(k-1,2-3)+Gu(k,2) (2-52) 


i=o j=o 
(i.j)# (0,0) 
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Let's define the set Q(k,£) such that for all i,j 
(Ei, 0J) ER(k,2), 
all the values of x(k,2) in &(k,2&) will be used to estimate 


Eme point x(k,$). 





-M 
hme Ry dy 
9-0 - 0 4 o o © 0-%--- -- or 
e . 0 ® 
K-Mı > 9 o 4 » 
o Pg 
5 
*. 
OK; L) 
E 4 
? A (x,25 e 
» x(k, 2—-M, ) o e @ 
o e e 9 0. e 
* e 6 9 9 0%... 9 


FIGURE 2-2 


DEFINITION OF 2(k,%) 
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Again the coefficients Ajj will be determined such that the 
mean squared error is minimized. The estimate of x(k,R), 


A 
x(k), is given by a linear combination of the previous values. 


x(k, 2) = - 45x (k-1, 2-3) (2-53) 
The mean squared error is 

Ele’ (k, 2) J=E{ [x (k,2)- x (Ky 2) 12} (2-54) 
Bit Ele’ (k,2)] is minimized, $(k, 2) is the "linear 


least squaresestimate" of x(k,2). 

Going through the same procedure as in the one-dimensional 
case, that is, substituting (2-53) in (2-54) and differentiating 
with respect to each EL setting derivatives equal to zero, we 
obtain the following set of simultaneous equations for the 
unknown Aij 

E([x(k,2) -5(k, 2) ]x (4,3) ) 2 o for all(ij)eQ, (2-55) 
which says that the coefficients Ajj must be such that the 
estimation error [x(x,2)-%2(k,2)] is statistically orthogonal to 
each x(i,j) that is used to form the linear estimate. This 
is known as the orthogonality principle in linear least square 
estimation. 

The modeling error is the difference between the true 
value x(k,2) and the estimate %£(k,2). By definition, 

er ei (Ben (K;0) (2-56) 
from the equation (2-52) and (2-53). 

Again, we are interested inttewhite noise field of zero 


mean andunit variance. Then the modeling error is also a 
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random field. 
Rewriting the Equation (2-52) in terms of the error e(k,2) gives 
x(k,2)=-2 2 A¡¿x(k-1,2-3)+e(k,2) (2-57) 
To calculate the error variance, multiply (2-57) by x(k-m,2£-n) 
and take the expectations 


E[x(k,2)x(k-m,2%—-n) J=- = DA¿¿Elx(k-1,2-3)x(k=m,2-n)] 


N 
FENEK x(k M,N) | (2-58) 

For the stationary process, 

E[x(k-1,2-j)x(k-m,2-n)]=R m-in-j], then (2-58) will be 

Ry Un) 77ZoZA; ;R, (7i n-3) EA 2959) | 

The second term on the nghtside of (2-58) is zero, because of the 
orthogonality principle and R(0,0) can be obtained by the 
following equation: 

e 2 

Ry y 0) ==2 2 A¡ ¿Bego 3) +6 (2-60) 

Therefore themdeling error variance, 


2 is given by the equation: 


Ele“ (k,2)]=E([ [(Gu(k,2)1%) = G 

Gó=E [e* (k,2) ]=R¿¿0, 0) +22 AisRy (33) and the mean of error 
e(k,2) is Ele(k,2)]=E[Gu(k,2£)]=0 

Again with the knowledge of autocorrelation function of the 
two-dimensional stationary random field, the filter coefficient 
Aj can be obtained from the Equation (2-59) and using these 
values of Ais: the gain G in Equation (2-52) can be calculated 
by Equation (2-60). 


Example 1 Consider a one-dimensional stationary band limited 


random process forwhichtheautocorrelation function is given by 
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Ry y (Mm) = PM cos (wom) 


Find a model which describes this process. An 
all-pole model is chosen such that 
P 


x(n =- y A,Xx (n-k) *Gu (n) 
k=1 


where Ele(n)] 0 
E[u(n)u(m)] = Ómn 
One has to choose the aderof the difference equation; here,asecond 
order model (p=2) is chosen. Then 
x (n) =-A, x (n-1)-A,x (n-2) +Gu (n) 
The problem shrinks down to that of finding Aj: A^, G with the 
given Ro. (m). 
From Equation (2-49) 
Ri) = = $ A, R{i-k) 
k=1 
RAL) = - Aj&jo) - A2BL=1) 
R42) = = ARD = ARO) 
Eutting this in matrix form 
-R(o) - R(-1) A] | R(1)] 
E DERG O) Bj m PM 


From Rxx(m) = A cos (w m) 

-1 Ey COS Wo Ay D cos we 

= Pcos w -=l A, E 02 cos 2 Wo 
o ! 


Therefore Ai - p cox w,(1-f? cox 2 we) —- 


1 - be cos” Wo 
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— 


Res E P^sin^?w, 
2 1 2 2 
=- P COS Wo 
From Equation (2-51) 


q? 


ii 


a = i Ay RL, (k) 


+ 
L + AVR (1)t A5R. (2) 


et E COS Wot A,P*cos 2 Wo 


ee P2cos2wo ( (1-P2cos 2 wo) >4sinZw.cos 2 Wo 
l - f?cos?w, l - P2cos? Wo 


Example 2 Givenastationary two-dimensional random field with 


Rxx(m,n) = pim| pin 


find the autoregressive model of this random field (most mono- 
chromatic imagescan be assumed to have this form of auto- 
correlation function).A first-order model QU l, M, = 1l) is 


chosen. Then equation (2-52) can be written as follows: 
Where u(k,2£) is white noise with zero mean and unit variance. 


Then, using Equation (2-59) 


l il 
R(m,n)-- um id A;;Rym-i,n-)) jerwallm,. N je Q 
(1,j)#(0.0) 
Rxx (1.0) = - Aj¿R, (0,0) - Aj¡R, (0 -1)-A,, Rxx(1:-1) 
Rxx(lsl) = - Aro Ro (0, 1) - A1]R,, (0.0) “A Rex (ls 0) 
Rxx (0,1) = - AjgR,,(71,1)- A 11 (71,0) -Agi Ry, (Or 0). 
Putting thisin matrix form gives 
-Rxx(0,0) -Rxx(0,-1) -Rxx(1,-1) s Fell) 
=Rxx(0,-1) -Rxx(0, 0) -Rxx (1,0) Al] RSC) 
-Rxx(-1,1) -Rxx(-1,0) -Rxx(0,0) And Rxx (0,1) 
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For thegiven auto correlation function above 
2 


-1 -P -f Alo f 
=p -1 -£ Ay _ p? 
-p2 -P -1 Any 5 
which yields 

Ag = 7f 

Ay, = P? 

Ag mn 


The modeling error variance or the square of the gainG* can “be calls 


culated by Equation (2-60) 


G? =R_ (0,0)+ e 4 NEL 
Rene i=0 j=0 1) xx 
(i.j)7(0,0 
0: Ag1 Ry 0711) + A, QR (1+ A1gRyx (1,0) 
= (1-p*)? 
Therefore, the complete model for Rxx = pimi etn! 
IP, LL) + Ee &) + Px (k-1,2) (1-2 )u(k,2) 
where E (u(k.2) ) 2 0 


Hk. 2) u(k-p,2=q)] = al 
3. Method of Filter Response 
Another method of modeling a linear stationary random 
process is based on the concept thatalinear random process is a 
result of filtering white noise through a linear filter. In 
Section B. of this chapter, the properties of linear systens 


have been discussed. 
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For the discrete system, it is known that the filter output 
power spectral density ig theZ transformationof theoutput correla- 


tion function. That is, 


A 
Gyy(Z) = 7 [Ryy (m) ] 
Gyy(Zı, Z2) = 3 [Ryy (m,n) ] 
and also it is noted that 


Bro (2a) 


A = AS UN ; Z^ ), 
where Gxx(Z), Gxx(Z1, Z2)is the input power spectral density 
function and H(Z), H (Z1, Z2) is the transfer function of the 
filter. Denoting the white noise input as uíln)oruí(k,2£) and 
the output of filter, which is a linear stationary random pro- 
cess we are going to model as x(n)orX(k,2), the problem 
can be stated as follows: For a given autocorrelation function 
ofalinear random process (field) R(m),(R (m,n), find a linear 


system such that when the input is white noise, the output of the 


filter gives a given autocorrelation function R(m), (R(m,n)). 







u (n) H (Z) 





x(k, 2) 
Given Ry (Mm), Ry (Men) 


u(k, 2) H(Z1, 22) 


EIGURE 2-3 


THE MODELING PROBLEM IN THE DISCRETE CASE 





If the input is white noise, then 


cuu z Const, 


Guu (21^ 22) = Const, 
Therefore, the solution for the required filter is to find a 
Bumetion H(Z) or H(Z], Zo) which satisfies 
5 


Goy (27 1 Z^) = Const -H(Z,, Z4)H (21, Zo 


G,, (2) = Const:H(Z)H(Z 
1) (2-62) 
where CZ); G, (2,5 Zo) is known through the relation 
Zee yee (i) 
eZ ee Sir, m,n) 
since RL, (mM), Ry, (m,n) is given. 
But forthe two-dimensional case, there is an inherent difficulty 
i l -1 -1 
in factorization of Gey (Z y Z 9) to H(24, Z24)H(24 E Z> JE 
Therefore, only separable functiorscan be modeled by this 
technique. 
Example 1 For a given stationary linear random process with 
R(n) = Sala COs (wo m), MO, T1 Mi. 
XX 
Find a difference equation which will give a random process 
wth autocorrelation above. 
SS WAR . (m)] 


E e? (1-02) . [-Z Pcos wot (1202) -771 


(1-2PZcos WotP7Z7) (1-2027+cos Wwo+ptz” 


COX Wo] . (2-51) 
2 


The second step is to find a factored expression for G (2) i.e. 


E -1l 
G 12) = G} (Z) . G} (Z `) 
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Assuming that Goy (2) has the form 


Gy (Z) - 82 (1-p^) BINE cuu (2-52) 


1-202 lcos wote z? l-2pz cos wote z” 


Comparing (2-52) and (2-51), a and b can be obtained as 


a = a 1=P cos wer + Ite cos De 
p= > l-p cos weto’ 1+ P cos Wot" 
From equation (2-50), if we choose H(Z)= azl t b 


1-20z ^cos wet ^2 ^ 


then the term c* (1-e^) in Equation (2-52) can be considered 


as Guy (2) 


Therefore 
(1-p*) 6% if m=0 
R u (m) = 
0 if m#0 


and the complete model is drawn block diagrem form in Figure 2-4. 


x(n) has given 
autocorrelation 


u(n) 







az l4b 


1-2Pz “cos w +p2z-2 






White Noise 
with 
variance 
=(1-p2) 2 







x(z) 





FIGURE 2-4 
FILTER FOR ONE-DIMENSIONAL BAND PASS PROCESS 
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To put the system's input-output relation into state equation 
form, it is defined for convenience as 

aie) = zl w (z) 
were w(n) is also white noise with same autocorrelation 
function 

R(m) = (1-P%0% if m=0 

ww 

0 if mx0 


then the transfer function is 


E X(Z) az + + b 
E Uy il 2,72 
1-292 COS Wot Z 

Defining 

X4 (z) = -p*z ix, (z) 

X, (z) = U(z) 

-l 2 -2 
1-2P cos woz `+p Z 

x (2) T a 
then 

x, (n) = -p° 1) 

„m = P Xo (n- 

X. (n) = x, (n-1) + 2p cos WoX. (n-1) + w(n-1) 
and 

Sen) = ax) (n-1)+bx, (n) = -e7%ax (n) +bx, (n) 
Putting the state and output equationsin matrix formgives 

N 
m) fo- pi x, (n-1)) fe] wa 
X. (n) 1 2 cosw, X»(n-1 J 
m -2 
zn = (-p a b) X1 (n) 


x. (n) 
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Example 2 The two-dimensional "band limited" discrete 


Markov process is defined by the autocorrelation function 


(m) m=-071,2.2. 
cosiw2nJ] n=0,1,2... then 


R¿¿ (mn) = 6 [90 cos (w,m)] p, 
the discrete power spectral density 
EO M2 _p 2 ANZ EZ.) 

65 (237 221) =6 (1-5 )u-P,) IN? 


where 
A(Z Z,_)=[-Z,Cosw,+(1+ 23277) CoswW, ] [-Z,CosW.+(1+ 2| -zz1cos wa] 
ee 29 1 1 Pi 1 1 2 2* (99, 2 2 
AA. DEE -1 2 -2 
B(Z, Ze 2pz4cos w,+P127) (1 2P 424 cos wy +P Z4 ) ] 


2 


-1 2 - 
[(1- 2 p4Z4cos w „+P525) (1- 29,2 > COS w,+052> B 


Putting AlZ,, Z5) in the following form 
Bene (aoc tb) 1 i (a.ze- +b.) (a.z.+tb.)] 
M e» A WES, Dee 
and comparing this equation and above A(2125); a1, 85; by ; 5, 


obtains 


HE im 2 / 2 

l 5( l pcos w, + Py + 1 + pcos Wi*pi ) 

( 1-p,cos W. + |; 2 + 1 + cos w.+ 2) 
P2 en V Pa 2*[2 

b,= Es 3-0,C0S w, + 2 l + COS W, + 2) 
ie Fe TP) en i viel 

b,= 1( l-p4,cos w4 + p 4 8 [1 + pycos yip, 
2° = P2 DD [2o oP 2 


= -] -1 
Let H(Z; 7 Zo) = (a]2] a aara 2 


fy 
l 
NT 


-1l 2_-2 
(1- 2284 2 1 loos Bart onc ae) (1 20422 COS w.+072, ) 
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then from (2-50) 


R (mn) =[ 9 (1-P,D (1-P,%) if n=o, m=0 
O if n20, m#0 








X(Z, ,Z2 ) 
U(2,, 2,) 


White Noise 


of variance 


26? (1-042) (1-& Ry „im, n) 


ini 
- "(p Losu m J£, Cosu n) 
FIGURE 2-5 


FILTER TO GENERATE BAND PASS RANDOM FIELD 


It is convenient to define. 
= -1, -1 
U (Zi ; Zo) = 27 Zo W(z,, Z^) 


W(k,2) is also white noise with the same statistics as w(k,2). 


Rz (Mm) = 6.7 (1-,7) (1-97) if n=0 and m=0 
0 if nž0 or m#0 
Then, the filter has the form 
-1 -1 -1, -1 
X(24, Zo) un ae +b) ( a7Z, + Bol 27 Zo 


W(Z4 Zo) _2. 


(ui 29. 2 1 loos Ww MZ 1 a a > Los wW o ) 


The following definitions are made: 
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DET 
Z) = -P1 Z1 N5(2,; 25) 
Z-1w(Z2 ,„ 2) 
E 
Sala Zz ~loos W, + 2, B 
E M n 
o E CAS 


10 


No (Zir 2,) = 


From the last three definitions one can writetheset of 
difference equations 
i 2 
N4 (k, 2) = 0 -P1 Ny (k,2- 1) +10 
w(k,2£-1) 


N^ (Kk, £) l 2picos w N5(k,£-1) l 


1 
EE 2 

x4 (k, 7) = Pi a4N4 (k, 2) *b4N, (k, £) 

New, additional definitions are made. 


Be 
A A E a 25! 


l 
10 X] (2]» 22) 
1-2P,25" cos Wo oz, 


From these definitions it follows that 


Pes 


E 2 
X(2,, 24) - (a42, " * b4) M, (2,, Z5) (2-55) 
hen 


N 
M, (k , £) il 2f,cos W, M,(k-1,2) iL 
X(k,2) = -P,a M] (k, 2) + boM, (k, 2) (2:356) 


Combining treseall together , the following form is obtained. 


41 





"2 / ( 
M, (k+1,2) 0 -P3 0 0 M, (k,2) 0 
| 2 
M5 (k*1,2) _ il 2P,cos Wo "P131 bi M, (Kk , 2%) O |w(k,2) 
2 
Ny (k,2+1) 0 0 0 Py N, (k,2) 0 
N, (k,2+1) 0 0 1l 2P cos wi N, Oe 1 
and 
2 
x(k) = [=D> a, ba 0 0 ] M, (k,2) 
Ny (k,2) 
N^ (Kk, £) 


Two methodsof modeling have been discussed. Some of the above 
examples will be used in later chapters. It should be noted 
again that the filter response method cannot ke used for the 


case where the autocorrelation function is nonseparable. 
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TIE ADAPTIVE FILTERS 


A.  NONRECURSIVE FILTER 
P Introduction 

Many forms of adaptive filters have been described in 
the literature, some of which have been shown to be optimal 
(or suboptimal) in certain applications. The special form 
of an adaptive nonrecursive filter developed by Widrow [1] 
is reviewed here to give some insights to the recursive adap- 
tive filter developed in next section. 

The filter to be considered here consists of a tapped 
delay line, variable weights whose input signals are the signals 
at the delay line taps, a summer to add the weighted signals, 
and machinery to adjust the weights automatically. The impulse 
response of such a discrete system is completely controlled 
by the weights. The adaptation process automatically seeks an 
optimal filter impulse response by adjusting the weights. 

Two kinds of processes take place in the adaptive filter: 
training and operating. The training (adaptation) process is 
concerned with adjusting the weights,and the operating process 
consists in forming output signals by weighting the delay line 
tap signals, using the weights resulting from the training 
process. During the training process, an additional input 


signal, "the reference (or desired) response," must be supplied 


to the adaptive filter along with the usual input signals. 


43 





This requirement may in some case restrict the use of 
this particular form of adaptive filter. An example illustrat- 
ing the use of the desired signal is the case of modelling an 
unknown system by a discrete adaptive filter as shown in Figure 
3-1. Here a discrete input signal x(n) is applied to an un- 
known system to be modeled. The discrete adaptive model is 
supplied with an input x(n). The output of the unknown system 
d(n) is compared with the output y(n) of the adaptive system. 
This system can self-adapt to minimize the mean square error, 
(throughout this thesis, the mean square error is chosen 
as tle performance measure), where theerror is defined as the 
difference between the output of the adaptive model and the 
desired signal (for this problem the desired signal is the output 


of the unknown system to be modeled). 


z-l E zł 


adjustable 
weights 


2 2 A dnt se Dan 


y(n) Filter 
Outpu 





x (n) Unknown System 





Input To be modeled 


FIGURE 3-1 MODELLING OF UNKNOWN SYSTEM 
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Then 


N 
y (n) T WiX (n-i) (3-1) 


yam) (3-2) 


e (n) 


Noting that equation (3-1) is the convolution summation, the 
sequence of weights{wi} can be seen asthe impulse response 
of the adaptive system. 


The weights w, are adjusted to minimize E (e^) : 


It will be shown that if the input and output signals of the 
System being modeled are stationary, the error signal has a 
mean square value which is a quadratic function of the weight 
settings. 

For the minimization of mean square error, the steepest 
descent method is used. Throughout this thesis , the terms 
"filter coefficient updating process" and "adaptation process" are 
used interchangeably and it is assumed that the input to the 
adaptive system and the desired signalare stationary random 


processes (orrandom fields for the two-dimensional case). 


2. Performance Surface, Gradient and the Wiener Solution. 
The input signals are weighted and summed to form an 
output signal [Équation (3-1)]. Introducing the vector notation 
TURN 
such that Wen) = [Wo , Wi, Wo +s - Wy] 


au X) - [x(n), x(n-1), x(n-2) . . . x(n-N)] ay 
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Then Equation (3-1), which describes the linear combination 
(operating process), can be written in matrix form. 
y(n) = Wx = x? (3-4) 
and e(n) = y(n) - d(n) 
= Wx - a(n) (3-5) 


The square of this error is 


e(n) = Wx xiv - 2a(n) WX + d2(n) ge 
the mean square error, the expected value of e? (n), is 


Ele?(n)] = (in - MR; +WR, W (3-7) 


where the vector of cross-correlation between the input signals 


and the desired response is defined as 


Bean) X (n) ] = E | d(n)x(n) 


Il 
I 


d(n)x(n-1) za 


d(n)x(n-N) (3-8) 
and where the correlation matrix of the input signal is defined 
as 
EIX(n)X (n) ] 2» EÍ x(mx(n) x(nx(n-1) . . . X(n) X(n-N) 


X(n-1)x(n)X(n-1)X(n-1). . .X(n-DX (n-N) SR. 


X(n-N)X(n-N 


= 
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It may be observed from (3-7) that for stationary input signals, 
the mean-square error is precisely a second order function of 
the weights. The mean square error performance function may be 
visualized as a bowl shaped surface, a quadratic function of the 
weight variables. Then the adaptive process has the job of con- 
tinually seeking the "bottom of the bowl." A means of accomplish- 
ing this by the well-known method of steepest descent is dis- 
cussed below. 

In the non- stationary case, the bottom of the bowl may be 
moving, and the orientation and curvature of the bowl may be 
changing. The adaptive process has to track the bottom of the 
bowl when inputs are nonstationary. The method of steepest 
descent uses the gradient of the performance surface in seeking its 
minimum. The gradient at any point on the performance surface 
may be obtained by differentiating the mean-square error function 
of Equation (3-5) with respect to the weight vector. The 


gradient is 


Mele mn) -2 Rg + 2 Ry Ñ Ba, 


To find the "optimal" weight vector Wi ms?" .i.e. the one that yields tte 


least mean square error, set the gradient to zero. Accordingly, 


Rya = Rxx “ims 


_ -1 (3-11) 
W us” Rex Rua 


Equation (3-11) is known as the Wiener-Hopf equation in 


matrix form. 
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Then the minimum mean square error may be obtained by 
substituting (3-11) into (3-7) 


ee A E 


LMS Rxa (3-12) 


ae MS Algorithm 


In seeking the minimum mean-square error by the 
method of steepest descent, one begins with an initial 
guess as to where the minimum point of the mean-square error 
surface may be. This means that one begins with a set of initial 
conditions for the weights. The gradient vector is then mea- 
sured, and the next guess is obtained from the present guess 
by making a change in the weight vector in the direction of the 
negative of the gradient vector. The method of steepest descent 
can thus be described by the following relation 


mires win) + Eviete” (n))] (een 


The expression for v[E (e? (n) )] is obtained by using 


Equation (3-10). 


W(n+1) = W(n) + AA - 2k R, (3-14) 


d 

The gradient vector VIE (e? (n))] is the gradient of the expectation 

of the squared error function when the weight vector is W(n). 
When the performance function is quadratic, the gradient 

is a linear function of the weights. The advantage of working 

with the quadratic performance surface lies both in this linear 

relation and in the fact that such a surface has a unique 


minimum. 
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The purpose of the adaptation process is to find an exact or 
an approximate solution to the Wiener-Hoff equation (3-11). 
One way of finding the optimum weight vector is simply to solve 
(3-10). Although this solution is generally straight forward, 
it could present serious computational problems when the num- 
ber of weights N is large and when input data rates are high. 
In addition to the necessity of inverting an Nx N matrix, this 
method may require as many as N(N+1)/2 autocorrelation and cross 
correlation measurements to be made to obtain the elements of 


R R 


xa” 

No perfect solution of equation (3-11) is possible in prac- 
tice to estimate perfectly the elements of the correlation 
matrices. 

A method for finding approximation solutions to (3-11) is 
presented below. The accuracy of this method is limited by 
statistical sample size, since weight values are found that 
are based on finite-time measurements of input-data signals. 

This method does not require explicit measurements of 
correlation functions, nor does it require matrix inversion. 

It is the "LMS" algorithm based on the steepest descent method. 
This algorithm does not even require squaring, averaging, or 
differentiation in order to make use of gradients of mean- 
Square error functions. 


When using the LMS algorithm, changes in the weight vector 


are made along the direction of the estimated gradient vector. 
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Accordingly 


- A 
W(n+1) = Win) + kV [Ele’(n))]} Fa) 


Where 


W(n) à weight vector before adaptation 


W(n+1) = weight vector after adaptation 
k 2 scalar constant controlling rate of convergence 
and stability (k«o) 
le estimate of gradient of E (e^ (n)] with respect 


to W with W = W(n) 

One method for obtaining the estimated gradient of the 
mean square error function is to take the gradient of a single 
time sample of the squared error; that is 

YE (e? (m)7 = V[e*(n)] = 2e(n) Vle(n)] (3-14) 

From Equation (3-4) 

Vfe(n)] = V{y(n)-d(n)] -v(W (n) X(n) -d(n)] 

UU (3-15) 
Thus, 


A > ES 
V(E(e” (n))] 2e (n) X (n) 


2 [W* (n)X (n) -d (n)1X (n) (3-16) 


The gradient estimate of (3-16) is unbiased, as will be shown 
by the following argument: For a given weight vector W(n), the 


expected value of the gradient estimate is: 


A 
E(V [E(e^(n))]) 2 2ECIWT (n) X(n) -à(n) ]E(n) ) 
E 2R, W= 2R -a (3-17) 
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Comparing (3-17) and (3-10), we see that 

El VIE (o2 ())1) -VE id (3-18) 
and therefore for a given weight vector, the gradient estimate 
Mie? 11 is unbiased. 

Using the gradient estimation formula (3-16), the weight 
iteration rule Equation (3-13) becomes 

W(n+l) = W(n) + 2k e(n) X (n) (3-19) 
and the next weight vector is obtained by adding to the present 
weight vector scaled by the value of error. This is the LMS 
algorithm. Looking at Equation (3-19), the adaptation process 
is a simple first order recursion equation which can be realized 


as shown below. 


WEIGHT 
SETTING 


X(n) 





e (n) 


FIGURE - 3-2 FILTER COEFFICIENT UPDATING 
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Tothis point, the basic concept of LMS nonrecursive adaptive filter 
has been introduced and reviewed. More details can be seen in [1l]. 
Widrow [1] showed that the weight vector mean converges to the 
Wiener solutionarnithatthe boundsofthestep size k should be in the 
region such that 


E D < k < o for the stability and convergence, 


max 





where A is the maximum eigenvalue of Rog: 


ax 
4. Two-dimensional adaptive filter. 
a. Adaptive filter structure 


The input-output relation of the two-dimensional 


filter is given by two-dimensional convolution. 


p q 
y(k,2) = E E: w- X(k-1, 2-3), (3-20) 
O O 


1] 

where y (k, £) is the filter output 

and ep the firiteimpulse response of filter. 

Here, it isassumed that the input. X(k,2) is a stationary random 
field. 

In Equation (3-20), a set of two-dimensional 
stationary input Signals is weighted and summed to form an out- 
put signal and the filter output is intended to match a desired 
(reference) signal in accordance with the minimization of mean 
squared error, where the error is the difference between filter 
output and desired signal. 


A 
e(k,2) 2 y(k,$ ) - d(k,2). (3-21) 
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Introducing the vector notation such that 


= 


[W_ Wol "OMM Nog Wio Wii POE Wig Woo oe pq 


and 


X = [x(k,2), x(k,2-1), ...x(k,2g)x(k-1,2)x(k-=1,2-1) ...x(k-1,2-9) 
x(k-2,2) . . . . . x(k-p,& -q)] (3-22) 
then Equation(3-20)can be written in matrix form. 


y(k,2) = WIX = XW SD 


where W is a weight vector of dimension (ptl) (qtl)xl 


>| 


is a input signal vector of dimension (p+1) (q+1)x1 

The weight vector of the filter is supposed to be adjusted in 
the direction such that performance criterion (mean square error) 
is to be minimized. Thus, the linear combinatorial system 


in Equation (3-20) will be given with variable weights. 





y(x, 1 
PIGORE 3-3 LINEAR COMBINATORIAL «SYSTEM IN 


TWO-DIMENSIONAL ADAPTIVE PILTER 
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In Figure 3-3, the linear combinatorial structure is given. Then, 


the adaptive nonrecursive filter can be drawn as following: 








LINEAR COMBINATORIAL 
OF FIGURE 3-3 


Input ¡WITH VARIABLE WEIGHT 
Stationary 
Random 
Input 


filter output | 


FIGURE 3-4 STRUCTURE OF NONRECURSIVE ADAPTIVE 
FILTER 


b. Wiener solution 
From equation (3-22) and (3-23), the error signal 
can be written by 
e(k,4) = WX - d(k,2) (3-24) 


The square of this error is 


e^(k,4) » WIxx!W - 2a(k,2) XIW + da (k) (3-25) 


The expected value of e? (k, 2) is 
2 - 2 y —I -T — 
Ele (k,2)]-E[d (k,2)] 2R. qW UE I (3-26) 


where the vector of cross correlations between the input 


signal and desired response is defined as 


54 


d (k, g) X (K, £) 
d(k, ££) x(k, 2-1) 


e. 


d(k, 2)x(k, 2-q) 
Eld(k,2)X] =E | a(k,2)x(k-1,2) | $, (3-28) 


Gale, 2) x(k-1, 2-0) 


d (k, L) x (k=p;, gg 
and where the correlation matrix of the input signals is 


defined as 


E[XXT] - 
BINE oe fog X1X (p+q) (q+1) 
E AG mmt X2X (p+1) (q+1) 
er %3%2 Kara tee a) (3-29) 


A 


En a+ D*+ aH) eee *t6 2) 622 * (p-2 63 | 


=R » b s = —p,2- 
Ser where Xt q+ x(k-p,£-q) 
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The gradient at any point on the performance surface can be 
obtained by differentiating the mean square error function of 
equation (6) 

Q E (e^ (k, £) 


V[E(e^(k,2) )] 
ds 


-2R.q* 2R -W 


To find the "optimal" weight vector W that yield the least 


LMS 


mean square error, set the gradient to zero. Accordingly, 


Red E Rex" 


= 
N 
ye) 
ye) 
rs 
As 


LMS XX (2309 

Equation (7) is the Wiener Eopf equation in matrix form, again 
the minimum mean square error is obtained by substituting 
(3-30) into (3-26). 


Ele (k,4) 1... Jn ANE prs aes (3-31) 


RNS Algorithm 
Consider a two-dimensional field x(k,£) to be processed 
(usually two-dimensional filters are used in process- 
ingdiscrete two-dimensional image fields) and assume that the 
two-dimensional field consists of NxN discrete points (which may be a 
sensed signal by NxN pixel elements ofa sensor). The adaptation 


processes is that of adjusting the filter coefficient Wis Zn 
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accordance of minimization of the mean square error as in the one- 
dimensional case. The adaptation scheme may be predetermined as 
being columnwise scanning or diagonal, or row-wise scanning. 


Here the row scanning process is adopted as shown in Figure 3-5. 





2 -q 2 2 
o e e 9 « 0 
e eo e. 0 
$ e 9 4 
5 = y el e 9 
f jt 3 
LNE l Region for(j-*1) 
y(k,2) = i , 
PE ai adaptation | 
E zwi; (k-iA-3)px “TL 341) Eh estimation point 
oo 
Den estimation point 
e(k,2) = e o4 0o 
uer -d (k, 2)b e na... 
= elj e 9 lad e. o 
ese .. e O 
e 9 9 eoe qeu. 9 ? 


FIGURE 3-5 ADAPTATION SCHEME 
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Therefore ,‚tienxNadaptation processes will be required to complete 
the filtering ofanNxN two-dimensional field. Using e(k,?) 

to denote E UE rcc Iu MIthorterotrzonctotWgesce(ctlr xs 
the (j+l)th error), then the filter coefficient updating process 
can be described by 


Wl) 2 W() * uVIE(e a (k, 2)] 


where W(j+1) coer ficient vector after adaptation 


W (j) coefficient vector before adaptation 


lic I » 


u negative scalar constant controlling rate 
of convergence and stability. 
The gradient of th mean square ervor is to be estimated by 

VIE (e? (k,2)] = Vle?(k,2)] 
where e(k,2) = y(k,2) - d(k,2) 
then W(j+1l) = W(j) - ue(k,2)V[e(k,2)] 

= W(j) - 2ue(k,2)X 

where X is defined by equation (3-22). 

Along with y(k,&) = WIX 
and e(k.£) = y(k,£) = d(k,2),; 


the LMS algorithm will ke completed. 


BEZERECURSIVE FILTER 
I- Introduction 
In the previous section, it is shown that adaptive non- 
recursive filters have a finite impulse response; that is, they 
can produce only zeros with no poles in the filter transfer 


function. This limits the capability of transversal adaptive 
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fitters in many applications. To overcome this limitation a 

new adaptive filter structure is described which is capable of 
producing poles in the transfer function. The basic configura- 
tion considered here is quite standard; that is, the present 
output sample of the filter y(n) is a linear combination of the 
present and past samples of the input x(n), x(n-1), . .x(n-M) 
and the past samples of the output, y(n-1), y(n-2). . . y(n-N). 
The present output sample of the filter is compared against a 
reference sample. The resulting error samples are used to adjust 
the filter parameters, feed forward gains and feed back gains to 
minimize some error function.TreMe-dimensional recursive filter 
is developed first, then it is Bee to the two-dimensional 
adaptive filter. 

Recently Feintuck [2] and White [3] have proposed a technique 
for making digital filters with zeros and poles adaptive. This 
development may enhance the possiblity of dtaining accurate 
models for unknown systems. The new approach is developed into an 
algorithm. It employs the steepest-descent criterion for para- 
meter adjustment but it differs in the estimation of mean 
squared error gradient vector from Feintuck [2] and Widrow [1]. 

2. One-Dimensional Adaptive Recursive Filter 

ar Structure 
The recursive filter is described by its transfer 


function 
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1 (3-22) 
In the time domain, the input-output relation of the digital 


filter is given by 


y(n) = i Exo aiy (n-i) o 
1=0 1=1 
where y(n) = nth sample of the filter output 
x(n) = nth sample of the filter input 
a; = feedback coefficients i = 1,2,. N 
bi = feed forward coefficients 1=0,1,2..N 


The output samples of the filter are intended to match those of 

a reference (or desired) signal d(n). In accordance with 

the minimization of some error criterion, the filter parameters 
ai, b: will be adjusted at every iteration Theceneral scheme of the 
adaptive recursive structure is given in Figure 3-6. The two 


finite length transversal filters areusedinthe forward path and 


feedback path to form the recursive filter of Equation (3-33) 
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d(n) 


reference signal 


input Signal ` i 


Erw EPTETERSI 


x(n) | 23 


| FILTER 2 





i 
| E 


EU | 
FIGURE 3-6 ADAPTIVE RECURSIVE LMS FILTER 


USING TWO TRANSVERSAL ADAPTIVE 


FICTERS 


b. Problem in Wiener Solution 


Introducing vector notation for the signals and sets 
of filter coefficients we haye 
= [ a1; Aor eee aN] 


B = [ bo bi, x NUT by] 


exl 
Z 
H 
il 


gsm (in-l1) « s. x (n-M)] 


Kl 
S 
rH 
il 


2 in-2) 5 yin-N)} 
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Then Equation (3-33) can be written as 
y(n) = B"X(n) -A* Yin) (9592) 
where Ais the feedback coefficient vector (nxl) 
Bis the feed forward coefficient vector [ (M+1)x1] 
X(n)is tre nput signal vector at nth iteration [(M-lxl] 
Y(n) isthe output signal vector at nth iteration (Nxl) 
The performance criterion is again minimum mean squared error, 
where the erroristhe difference between filter output and desired 
Signal (reference signal). 
That is, the filter is used to estimate a desired waveform 
d(n) in a minimum mean square error sense. Assume that the 
Observables are stationary and zero mean and let e(n) denote the 
error waveform at nth sample, then 
e(n) & y(n)-d(n) (3-35) 
- BiX(n)-AlY(n) -d(n) 
and the mean square error is 


2 


Ele? (n)] E [ (B^ X (n) -A3 Y (n) -d (n) ) 


] 
= E(B? X(n) X(n) -B-2B- X(n) Yin) AtA-Y¥(n) YA 


-28la (n) X (n) »2ala (n) Y (0) 47 (n)] 


EJ = mem an 
A'"R,,A-2B'R 
Do x B 


u 2 =T = 
= Elda” (n) ] +B p DE y 


—T 


—T 
E Rax* 2^ Ra (3-36) 


Y 
pere R = Pape Ca 


Ruy = E[Y (n) Y7 (n)] 


T Eld(n)X(n)] 
Ray == Eld(n)Y(n)] 


and E = E[X()Y. n 
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The theory of Wiener filtering employs known second-order input 
statistics to dictate the impulse response of the linear filter 
that minimizesthe mean square error; that is, as in the previous 
section, the knowledge of second order statistics Ray! p is 
assumed to calculate the optimum impulse response 

(optimum weight vector in nonrecursive adaptive filter) Wy,  . 
But here in the recursive algorithm, it is also required that 

the autocorrelation of the output, RL. and the cross correlation 
of the output and the input R y ind the cross correlation of the 
output with the desired waveform,Rą Should be assumed known. 
Thus, the set of statistics mentioned above is assumed to be known 
for a moment, and will be used to determine the weights in 

the recursive filter. The statistics for the fixed para- 
meter filter are not a function of these statistics, but instead 
the weights are a function of these statistics. Therefore, Rey’ 


Rax 


is made with respect to A and B. 


and Ruy are to be considered constant when the differentiation 


The set of weights (filter coefficient vectors) which minimize 
the mean square error can be found by getting the gradient vector 


with respect to filter parameters equal to zero. 


. 2 
ue c RA AMS 2-R "BC Re = 0 
3 RÀ 27 Xy dy 
- -1 
A = R (R, -R B 3-37) 
YY dy xy ^! ; 


and 


2 
3 Ele” (n)] A VE 
a 2R B- 2R A-2R 
XX Xy dx 


pM > Rey ® (3-38) 


UJ 
N 





Thus, one can solve for the filter coefficients if all the 
second order statistics are known. But without knowing the 


impulse response of the filter, the R fe and a not be 


y dy 
calculated with only input and reference signal statistics. 
Noting that we are looking for the impulse response which 
minimizes the mean square error in some way, it is clear that 
Ray! Ray’ R „are not available and so the Wiener approach is 
not feasible. 
C. LMS ALGORITHM | 
An iterative gradient search technique (the method of steepest 
descent) is revisited, Here, in the recursive algorithm, it 


updates the filter coefficients with steps proportional to the 


gradient vector. This updating process is 


Aln+tl) = Aln) + k AA 
XU k V, [E(&(n)) ] (3-39) 
B(n+1) = Bín) + k, AB 


B(n) + k, vg [E (e^ (n) ] 
where 


A(n), B(n) a filter coefficient vectorsbefore adaptation 


yA 


Ex gt i) Filler coefficient vectors after adaptation 


n 
= 


k Ky Scalar constantscontrolling rate of convergence 


ar 


and stability (k_, k, <0) 


gradient vectors with respect 


D> y 


Va [E(ef(n))], Vg EECe^ (0))] 


to A and B respectively. 
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The updating process (3-39) can be considered as afirst order fil- 
tering process with an input proportional to theq@adient vector. 
But the gradients VA[E(e^(n))] and Vg([E(e^(n) ) ] should be esti- 
mated because the output statistics are not available apriori or 
an infinite statistical sample would be required to estimate 
perfectly the elements of the correlation matrices in Equatiors 
(3-37)and (3-38). A method of estimating these gradients will 

be presented. 

Widrow [1] obtained the estimated gradient of the mean square 
error function by taking the gradient of a single time sample of 
the squared error (instantaneous estimates) when he discussed 
Enesnöonrecursive adaptive filter (see previous section). 

Here, in this thesis work, a new method of estimating 
gradients is proposed. This is to approximate the mean squared 
error E (e* (n)] by an average of a finite number of points at every iteration 
and take the gradient ofthis instead of taking the instantaneous 
error square, that is, the approximation used is 

l L-1 


2 
E(e"(n))—— 2 
b $20 


e^(n-1) (3-40) 
For E e (n)), the average of the square error for the previous L points 
is taken and then gradient is evaluated for the approximate mean 


square error. The estimated gradient of mean square error is 


V E[e (n)] MI ON 
£=0 
Y Ele“ (n)] =V y [ LES a] (3-41) 
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For convenience, thetl ) term in (3-40) was dropped. Introducing the 
vector notation for error signal 
en) = e(n) 


e(n-1) 


e(n-L+1) 
then it is seen that the error signal vector is an (Lxl) vector. The 


estimated gradient (Equation (3-41))can be put into the matrix form 


V,Ele“(n)] = v,1 Em) Z(n)] 


VgEle^(n)] » Vg €T (n) €(n)] 


Substituting the estimated gradients in Equation (3-39), the updating 
process for the filter coefficients is: 

A(n+1) = A(n) * K,V, [£7 (n) € (n)] 
and 

g(n*l) 2» B(n) * K Vale (mE (n) ] (3-42) 


The function €! (n) En) is a scalar function of the coefficient 


vectors A and B, that is,. 


€T(n)E(n) - £(R, B) 


Therefore , by definition, the gradient of f(A, B) with respect 


to A and B is 
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Ob 
O 
af of _ 3f 
i E 
9A oB 
EH 
ob, 
It follows that 
0 ,fE(e-(n)¥] = v,lET(niän)] 
—T' =; 
ðo (e€ (n)e(n)) _ 
= a] "ge emo 
go 
—T' = = 
ðo (e` (n)e(n)})) m oc (n) 
en | an 3-43 
9a» 2€ (n) a, ( ) 
E = E 
ale (1) — I dE (n) 
ay | 2E l- 
and 
Vg (E(te^ (n) 2) 7 v4 (€ (n) TE t) ) 
= T = =; 
ocu E€ (n) D = de (n) ] 
en s = D € (n REI ELT 
= mo = 
9g (cum e (n) —, T de(n) 
3b, 2 e(n) 8b, 
(3-44) 
PPM A 
OE) ET) —, T de(n) 
2€(n) ———— 
PEN J ¿y 
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Consider the terms de(n), d€(n) in equation (3-43), (3-44), 





mere p=1,2,.+..N and q = 0,1,2,...M. 


Since €(n) and e(n) are defined as 


e(n) 4 y(n) - d(n) 


"c (e(n), e(n-1), . . . e(n-L-*1), 





de(n) =| de(n) 
da a 
p p 
Peal p p (3-45) 
and oe(n) 
q 
de(n+1) = 
= a012, eM 346) 
omm) _ Pd 
a 





dy (n-L+1) 





Note that3e(n) and de(n) are (Lxl) vectors 
Jap obq 


and E (n) is OMR) Vector. 
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^ ^ 


Therefore, VA [E (e^ (n))] and V, [E (e^ (n) )] in Equation (3-41) 


are (Nx1) and [(M+1)x1] vectors respectively. 
Equations (3-45) and (3-46) may be considered as sensitivity 
vectors which tell how much the change in Gp and bq affect 


the outputs y(n), y(n-1) . . . y(n-Ltl). 

TO calculate the elementsofthe estimated gradients of mean square 
error in equation (3-41), we should calculate the sensitivity 
vector of equation (3-45) and (3-46) first. 


From the recursive equation (3-33), 








dy (n) a M N 
eo | I bX(n-i) - I A;y(n-i) 
ddp dap i-o i-1 
N 
- - y(n-p) - I a, Zi (3-47) 
i-1 9 dp 
p 1, 2, EN 
dy (n) = x (n-q) = À a dy (m-i) 
bg j= a abd 
O LM (3-48) 
The sensitivity vector components ' given by Equations (3-47) ard 


(3-48) can be interpreted as being the response of a linear system 
with transfer function. 


1l (3-49) 
Henceforth, ,this will be called the "Sensitivity filter." Equation 
(3-49) is mall pole filter (recursive filter) with input signals 


[-y(n-p)] and [x(n-q)] respectively. 


69 





Now, what are ehe initial conditions characterizing the re- 
cursive relationships of the sensitivity filter? 

x(n-q) is the q time units delayed signal of input x(n) to the 
adaptive recursive filter of Equation 0-33)and y(n-p) is the p 
time units delayed signaloftheoutput y(n) of the recursive 


filter. 


Brom X(n) O 
y(n) = © hom =< O) 

x(n-q) and y(n-p) are sequences with the first q elements and first p 

elements zero respectively.And since changes in the ap and bq coeffici- 


ents have no effects on the system's response until n=p and n=q 


respectively, it follows that the initial conditions are: 


pm) 2E 
BED =O tob neos se. pri 


apar = O Gorensc c aL 
d 


A summary of this algorithm is 
the following: 
l. Calculate the sensitivity vector components.through the 
sensitivity recursive filter by equations (3-47) and (3-48). 
2. Calculate the estimated gradient by equation (3-43) and (3-44). 
3. Calculate the filter coefficient vector by equation (3-42). 
4. Calculate the filter output by equation (3-34) 
5») Form the e(n) vector €(n), then go back to the lst step. 
Note that due to the fact that the gradient of finite point square 
error average is used for the estimation of the true gradient of 


mean square, this filter cannot give an optimal solution, but the 
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more averaging points are used, the better performance is expected. 


It shouK be noted that if L=l, that is, 


Va [E(ef(n))] = vafe" (n)] 


V4 [E (e^ (0))] 


len] , 

this corresponds tousing the instantaneous error square for 
estimatingthe gradient, and this filter reduce s to the adaptive 
recursive filter proposed by White [3].Ifthefurther approxima- 
tion is made that the sensitivity components of equation 
(3-47), (3-48) are 


dy (n) 


Sap ~ ya 
u 


then the estimated gradient is 


^ 


V [E (e^ (n) )] = - 2 e(n) | y(n-1) = -2e(n)Y 
y (052) 
and 
ValE(e?(n))] = 2e (n) fxn) 
x (n-1) =2e(n)X 
x(n-N) (3-51) 
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Then the filter coefficient updating process is 
A(ntl) = A (n) -2k, e (n) Y 
B(n+1) = B (n)+2k, en) X (3-52) 
where koe ke 0 and filterTouüutpüt 


y(n) = B(n) !*X-&T (n)Y (3-53) 


Equations (3-53), (3-52), (3-51), (3-50) are exactly the same as 
the algorithm proposed by Feintuck [2]. This Feintuck algorithm 
has an advantage in simplicity when compared with the algorithms 
proposed by White [3] and proposed here which require additional 
recursive filters to generate the estimates of the gradient. 
Thus, it may be useful to extend the Feintuck algorithm to the 
two-dimensional recursive adaptive filter for simplicity. In 
the next section, the algorithms proposed by White and Feintuck 
are extended to the two-dimensional algorithm. 

3. Two-dimensional Recursive Adaptive Filter 

In this section, a mathematical model of the adaptive 
recursive filter for the processing of two-dimensional signals 
1s proposed. This can be considered as an extension of Fein- 
tuck's algorithm to two-dimensional filters. 

Two transversal filters having the same structure as the 
linear combinatorial system used in the non-recursive two- 
dimensional processor, are used in the recursive processor, 
one for the feedforward path and one for the feedback path. 

The two-dimensional recursive filter is described by 


its transfer function. 
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X ( ) = FM ont) 
ZZ = Z Z 
l 2 pu en SI 2 
"oz Za) 
qu 2 N -m -n 
Low T Lo. mn?1 22 
m=o n=0 
(mip) + (0, 0) 


In the spatial domain, the input-output relationofthaligital fil- 


ter is given by 


p q M N 
y(k,2) = z L b..x(k-1,£-j) - 2 L a. y(k-m,£-n) 
i-0 j=0 7 n=0 no ™ 
(3-54) 
The following notation is introduced 
=T _ 
BU = [bon Bo] E. Pig bii LL. bp ME gos Doc 
x = [X (KR) ,x(k,2-1),..x(k,2-q)x(k-1,2)x(k-1,2-1) ...x(k-1,2-q) 


ae en med en Klk-p, £-q) 1 
and A = [51 892 --&g 81 11 -c?1889207 ccc 7 ayy] 
Y =[y(k,2-1), Y(k,2-2) ...y (k,2-N) y(k-1,2)y(k-1,2-1)...y(k-1,2-5) 
AN a a Y (Kk=M,L-N) J 
The filter coefficient vectors A and B are  [(M+1) (N+1)-1]x1 
and (p+1) (q+1)x1, respectively, and the input-output signal vectors 
are again (p+1) (q+1)x1 and [(M+1) (N+1)-1]x1, respectively. 


Then equation (22) can be written as 
y(k,2) = B*X-A "Y (3-55) 
Here, to obtain an estimate of gradients of the mean square error 
function, a single sample of the square error is taken. That is: 
v [E(e? (k,2))] ee], and again the 
adaptation scheme (filter coefficient updating process) is used 


in the same fashion as in the nonrecursive case [see Figure 3-5]. 
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Denoting the error at jth iteration as e(k,&) then 


eh) SS) re 2k ea) 7, (605) ] 
B (j+1) = B(3) + 2k,e(3)V, (e (3)] (3-56) 
where 
e(k,L) = e(j) $ y(k,2) - d(k,2) (3-57) 


The componentscofthevectorsV, [e(k,2)] andy, le(k,21] can be 


A 
calculated as following. 
From Equation (3-57) 


Y, le(k,2)] =0, [y (k, 4) ] 


uen ale 
201 aoM EO m 2220 
A cc cue oy (k, 2) T 
JAMN | (3-58) 


and Ve [e(k,2)] =V AG 


Dee I IV(k,2) IV[ÍKk,2L) -.. dy (K, L) ay (k,£2) 


SE Dr Sb  — C9 
OO oq 10 lq 20 


ME E Ud EE ay (k,&) y 
¿Bog (3-59) 


Note thatV, [e(k, 2) ] and Vy [e(k, 1) ] havethe same dimensions as A and 
that is: 
[ (M+1) (N+1)-1] x 1 and [(p+1) (q+1)], respectively. 


From the recursive relation of Fquation (3-54) 


M N 
Yu) - -y(k-r,2£-s) - 2 za” ay mpm) 
rs m-o n-o rs (3-60) 
- M N 
and ày(k,2) = x(k-u,y-v) à A 90 y(k-m,£-n) 
Iro n-o mn _ 
uv (m,n) #(0,0) uv (3-61) 
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The recursive relationship of pquations (3-60) and (3-61) should 
be noted again, which can be implemented by additional recur- 
sive filters. 

Forming the instantaneous error gradient of Equations 
(3-54), (3-55) using the output of additional recursive filters 
of Equations (3-60) (3-61), the filter coefficient adaptation 
process of Equations (3-56) (3-57) can be performed. Note that 
this algorithm corresponds to the two-dimensional version of the 
algorithm proposed by White [3]. 

If we make the approximation 


dy (kK,2) 


>= y(k-D,2-s) 
dao 
yk A) | x(k-u, y-v), 
uv 


then it follows that 


le, 0,21) A 


a 


y (k, 2-2) 


ll), 
var IN) 
y (k-2,2) 


y (k-M, £2 -N) 
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and Vgle(x,2)] = | x(k,2) 


x(k,X—q) 


x(k-1,2) 


x (k-p,i-q) J 


Therefore, in this case, the complete algorithm is described ky 
A 
e(k,2) = e(j) = y(k,2)-d(k,2) 


A(j+1) = Alj)-2k,e(j) Y 


Bere e (j) X 
and Y(k,2) - B7X - AY 
This is the two-dimensional version of the algorithm proposed 


by Feintuck [2]. 
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IV. ADAPTIVE NOISE: CANCELLER 


A. THE CONCEPT OF ADAPTIVE NOISE CANCELLING 

Noise cancelling is a variation of optimal filtering that 
is highly advantageous in many applications. Specially in 
Wiener filtering or Kalman filtering, which are optimal, 
apriori knowledge of both signal and noise statistics are 
required. Adaptive filters, on the other hand, have the 
ability to adjust their own parameters automatically, and their 
design requires little or no apriori knowledge of signal and 
noise statistics while the Wiener approach utilizes a fixed 
parameter filter based on known statistics. 

Figure 4-1) shows the basic problem and the adaptive 
noise cancelling solution to it. It makes use of a reference 
input derived from one (or more) sensors located at the points 
in the noise field where the signal is weak or undetectable. 
This input is filtered and substracted from a primary input con- 
taining both signal and noise. As a result the primary noise 
ls attenuated or eliminated by cancellation. 

At first glance, subtracting noise from a signal seemsto 
be a dangerous procedure. If done improperly it could result 
in an increase in output noise power. If, however, filtering 
and subtraction are controlled by an appropriate adaptive 
process, noise reduction can be accomplished with little risk 


of distorting the signal or increasing the output noise level. 
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FIGURE 4-1. THE ADAPTIVE NOISE CANCELLING CONCEPT 
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FIGURE 4-2 NOISE CANCELLING WITHOUT AN EXTERNAL 
REFERENCE SOURCE 
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The following argument for the above is mainly due to 
Widrow, et al [4]. In Figure (4-1), a signal S is transmitted 
over a channel to a sensor that also receives a noise N, 

A second sensor receives a noise 
Ny uncorrelated with the signal but correlated in some un- 
known way with the noise Nj. In addition to these noises, 
additive random noises Mo and M uncorrelated with each other 


i 
and with S, No and Nare present. Then the reference input 
is 
d = s+ DESEM (4-1) 


and the primary input 


x = Nt M4 (4-2) 
The noise N, + M is filtered to produce an output y that is as 
close a replica as possible of No + Mo A This output is 


subtracted from the reference input S + No + Mo to produce the 
system output 


Z= Sot No + Mo cuv 


In other words, the practical objective: of the noise cancelling 
system is to produce a system output z = S + NONE SL. Y that is 
best fit in the least square sense to the signal S. This ob- 
jective is accomplished by feeding the system output back to the 
adaptive filter and adjusting the filter throughthe LMS 

adaptive algorithm (described in previous chapter) to minimize 
total system output power. Note that the system output serves 


as the error signal for the adaptive process. 
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Assume for the moment that the noises My and Mi do not exist, 

then if one knew the characteristics of the channels over 

which the noise N is transmitted to the reference input, it 
would be possible theoretically to design a fixed filter capable 


of changing N, into Noo That is, if the correct model of this 


Ji 
transmission channel, H(z), is obtained, the adaptive filter 
would be simply 1 / a fixed filter. 
H(Z 
Assume that S, No’ Ni, Mor Ma, and y are statistically 


stationary and have zero means. Assume that S is uncorrelated 


with No and N,anithat Mani M, ae uncorrelated with each other and 


1l 


ween S, No and N 


l' and suppose that N, is correlated with N 


l 
The output z is 
z=5 + No -+ M = y (4-3) 


squaring, one obtains 


DNE. 2 = 2 _ S 
Zo) F (No + My pU (N, + M y) (4-4) 


Taking expectations of both sides and realizing that S is un- 
correlated with No , M and y, yields 
pom 2 2 2 
Elz | EIS ] F E [ (NS + Mo y)^“] +2E[S( No + Mo y)] 


li 


E[S^) + ELN, +M- y) (4-5) 
The signal power ES] will be unaffected as the filter is ad- 
justed to minimize E[z^]. Accordingly, the minimum output power 
is 
Dee t. 2 2 
min E[z ] » E[S"] + min EIUS + M5 - y )^“] (4-6) 
Since the filter is adjusted so that E (z^) is minimized, therefore 


E [ (No EM = y) *] is minimized. The filter output y is then a 


80 





best least squares estimate of the noise N T M. Moreover, 
when E [ANS + Mo E is minimized, E[(z-S) *] is also mini- 
mized. Since, from (4-3) 


2 A Ey (4-7) 


Adjusting or adapting the filter to minimize the total output 
power is thus equivalent to causing the output z to be a best 
least square estimate of the signal S for a given structure and 
adjustability of the adaptive filter and for the given reference 
input. The output z will contain the signal S plus noise. 

From (4-3), the output noise is given by ue IS y). Since 
minimizing the E[z^] minimizes the EIN +M- y)2], minimizing 
the total output power minimizes the output noise power. Since 
the signal in the output remains constant, minimizing the total 
output power maximizes the output signal to noise ratio. Note 


27 = Ocan be achieved , then E [z?] E E(S*), 


that if E[(N, - y) 
therefore y - No and z = S. In this case, minimizing output 
power causes the output signal to be perfectly noise free. Also 
note that, on.the other hand, when the reference input is com- 
pletely uncorrelated with the primary input, the filter will "turn 
itself off" and will not increase output noise. 

In this case, the filter output y will be uncorrelated with 
the primary input. The output power will be 

E[z^] - E[(S* M, * NO] - 2Ely(stN, * M. )] * E[y*] 


- E[(S & M, 4 M) ?] * E[y?) (4-7) 
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Therefore, minimizing output power requires that E (y? 


) 
be minimized, which is accomplished by making all weights 
Zero, bringing E [y^] to zero. 

It should be noted that in applying adaptive techniques 
to a practical systems problem, the key step lies in providing an 
appropriate desired response signal for the adaptation process, 
that is, the reference input should be provided through the ap- 
propriate scheme, while the exact knowledge of statistical 
characteristicsare not required. In adaptive modeling applica- 
tions, the desired response is generally available as the output 
of the unknown system to be modeled. And also in the noise 
cell ing scheme above, the reference input is available by 
sensing noise which is correlated with the noise at the primary 
input in some manner. 


In next section, the signal filtering problem is discussed 


when no external reference input free of signal is available. 


B. NOISE CANCELLING WITHOUT AN EXTERNAL REFERENCE INPUT 

This section 1s concerned with signal filtering 
(estimation) a noise-corrupted signal when no external 
reference input is available. Here, it is assumed that only the 
noise corrupted signal is available, that is, referring to the 


Figure (4-1) of the previous section, the noise free of signal N, 
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which is correlated with No that corrupted the signal S is not 
available; only S + N is available. 

It is proposed to estimate the signal S by cancelling the 
noise Ns in some adaptive way. In the following, it is shown 
how a reference input can be obtained for the adaptation process 
under certain conditions. Assume that the noise corrupted 
“signal x = S + N is composed of broad band noise N and a narrow 
band signal S, then the autocorrelation function of the signal 
is broad and that of the noise is narrow. Also assume that noise 
N is uncorrelated with the signal and that the mean values of 
both signal and noise are zero. 

Consider a signal delayed by $6 units, 

SO SS == 0) Em (0. = 5) (4-9) 
where ô is a sufficient number of time units so that the noise 
component is decorrelated, but the signal component still 
remains correlated. 

Then 

Elo na (J79) | = 0 

ESA) S (5) 1 55 0, finite (4-10) 

For the two-dimensional signal, a signal delayed by 615 95 
units in the horizontal and vertical direction respectively, 
where $1 and 6, are sufficient length of spatial units such that 


the noise field would be decorrelated but the signal field still 
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remains correlated, then 

E[N(k,2) N (k-64,2-65)] = 0 

EÍS(k,2) S (k-64,£-765)] # 0, finite (4-11) 
and again it is assumed the signal field and noise field are not 
correlated with each other. 

Now if this delayed signal is used as a primary input and 

the original input used as a reference input to the adaptive filter, 
then referring to Figure (4-1) of previous section, 
55 =0) or S(k-6,,2% 07) can be considered as N,in Figure (4-1) 
and N(j-$) or n(k-9,,2 - 6,) as M,, and S(j) or S(k,2) can be 
considered as No and N(j) or N(k,4) as Min Figure (4-2), re-: 
spectively. 

From equation (4-10) and the assumptions that the signal and noise 
are uncorrelated, it is seen that the assumptions made in the 
last section for the various signals holds here. 

Therefore, from the argument in section IV-A,, the 
eater output would be a good estimate of the signal S, Figure 
4-2 shows the noise cancelling (or signal estimation) scheme 


Ascussed above. 


84 





V. EXPERIMENT AND RESULTS 


In this chapter, a computer experiment is performed to 
check the feasibility of the algorithms derived in Chapter 
IT for certain applications. The signal estimation problem 
for a noise corrupted signal is treated here for both one- 
dimensional and two-dimensional cases.  Nonrecursive adaptive 
filtering and recursive filtering have been examined and the perfor- 
mances of adaptive filters are compared to that ofthe optimal 
Wiener solution. The adaptive noise cancelling scheme is 
used for this application. 

First, consider a band limited one-dimensional signal S 
corrupted by noise N; it is desired to estimate the signal. 
If the statistics of both signal and noise are known apriori, 
a fixed optimal filter to estimate the signal can be designed 
by the Wiener Hopf solution of equation (3-11). 

Here it is assumed that these statistics are not known 
apriori but only that the signal is narrowband and 
the noise is a broad band signal and the signal is entirely 
uncorrelated with the noise. Then the signal has a wide cor- 
relation function while the noise has a narrow correlation 
function. Separation of this broadband noise and narrowband 
signal is now required for the estimation of the signal. 

It is assumed further that thedesired (or reference) signal 
which is needed for the adaptive vrocess is not available, that 


is, no other possible reference signal is available which nay have some 
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correlation with the signal we want to estimate, 


This problem can be considered as an adaptive noise 
cancelling problem without reference input. Assuming that the 
noise is white, then from the Figure (4-2) one unit delay is 
enough to decorrelate the noise component appearing in the adap- 
tive filter input from the noise camponent in the desired signal. These 
components will thus appear in the error but not in the filter 
output. The narrowband component, on the other hand, will not 
be decorrelated by the delay and will appear in the adaptive 
filter output. 

The input signal would be 
NO 
where S(j) bandlimited signal 
N(j) white noise 
and the reference input would be 
Gas Gist oN (j=l). 
The form of autocorrelation function of the signal is assumed as 


R(m) = pm 


COS Wg m 
For the purpose of computer simulation, the following values 
are assigned: 

p = 0.95 

We= 0.025 


and the variance of noise is 0.5. 
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For the optimum filter design using thealbvevalues,atransversal 


filterhavingl0delays was used. From equation (3-11). 


W. -1 


LMS = R R 


XX xd 


The autocorrelation matrix De WaS computed as 


1.50030 0.9970 — 0.90137 0.85495  C.81C44  C.7677% 0.72684 0.68767 0.65020 0.61436 
9.949790 1.50000 0.94970 0.92137 0.85498 0.81044 0.75774 0472684 0.63747 0.65020 
EELUDIST — 0.95973 .— 1459300 3.594975 7 0500137 — 5:85495 ^ 0.81044 — 0.75774 0.72584 70.587617 - 
—0.85496. 0.90137. 0.94970. 1.50000. . 0.94570... 0.90137 —.0.85496 .. 9.8104*. 0.75714 0.72684. 
0.8104» — 0.85496 0.50157 0254979 1.5929) 0.9497) 0.32137 0.85496  O.81u44  O.TET14 
"Q.i61!«  O.dlüs« —C.85«95 0.90137 6654570 1.59092 0.94910 0.90137 0.85496  QO.81044 
ENOLus —0. 75779 0.01944 —0.35496-7-.0.9C137 -—0.9«970 — 1.50000 0.94970 ^ 0.97137  0.85456€ 
EN UOCE 0.72084 3.76774 0.813644 .. 2.95496 _. 0.90137. 0.94970 — 1.523300 0.9497C — 0.90137 . 
0.65920 — Q.t8le] 0.72684  0.7677« — C.81044 — 0.85496 — 0.32137 | 





— 











ro wo 


3.61536 0.6592) 0.687o7 0.726584 2.76774 0.81044 9.85455 0.99137 3.5497€ 1.512332 





and the crosscorrelation matrix Ra as 


T 
Rea 


E 0.95979 0.99137 0.85696 1$ 04881044. D. T6114. 0.720684 — 0.687€7 0.65920 -0.6153t€ 











mn — a u —— P _— 





— - (———— es - 5 — — — en ._ —— — — - — 


Then the optimum Wiener Hopf solution gives the filter co- 


efficients as 


0.33297 
-0. 21191. 
0.13475 
0. 0855% 
0.05427 
—de034 36 —— 
J.92169 
0.01369 
9.003797 
: 0.00307... 


W = Wiener Weight-Vector - 











87 


) 





For simulation of the bandlimited signal, the state and out- 
put equatinns of example 1) in Section 2 of Chapter II are 
used. 

For the nonrecursive adaptive filter application, again 
10 delays and = -0.005 as a step size in LMS algorithm were 
used, and 2 delays for both feedforward and feedback path and 
EU tocas (equation(3-42) ), were used in the recur- 
sive filter application of both Feintuck's algorithm and the 
algorithm developed here. Eight points were used for error 
square averaging for the gradient estimation (L8, in Equation 
(3-40) ). The experimental results are plotted in the following 
along with the descriptions and optimal solution for the purpose 
of comparison. The results indicate the adaptive recursive 
filter appears to perform as well as the optimal Wiener filter 


once it reaches a steady state condition. 
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EIGURESR-I SIGNAL STATISTICS 


Autocorrelation = 0.96 M cos(0.025 m) 








NOISE CORRUPTED SIGNAL 


FIGURE R-2 


NOISE CA WAITE: 


Zero Mean 


* Variance = 0.5 








FIGURE R-3 WIENER-HOPF FILTERING 


10 Delays are used. 





FIGURE R-4 





WIDROW'S NONRECURSIVE ADAPTIVE 
FILTERING 
Number of delays used: 10 


Stepsize Used:  -0.005 


9? 
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FIGURE R-5 FILTERING BY FEINTUCK'S ALGORITHM 
NUMBER OF DELAYS IN FEED FORWARD PATH: 2 


IN FEED BACK PATH 2 


STEPSIZE USED:  -0.001 
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FIGURE R-6 


FILTERING BY THE ALGORITHM USING A 
FINITE POINT MOVING SQUARE ERROR AVERAGE 


For the estimation of gradient 


Number of Delays in Feed Forward Path: 2 


Stepsize Used: 


in Feed Back Path 32 


-0.001 
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As a second application, consider an image sensed by an image 
sensing device ofanNxN sensing elements array. It is assumed 
that this imageis composed of correlated background and a 

three diagonal line target trajectory. This image may 


be interfered withby the internal noise of device (assumed white). 


Then the output image includes three types of processes: 
x(k,2) = S(k,2) + T(k,£) + w(k,2) 

where 
S(k,2£) = correlated background 


T(K) Target strength (three diagonal line) 

W(k,2) = noise. 
Again it is assumed that no statistics are known apriori 
and the correlated background is a PER EOWBAHA signal. It is 
further assumed thatthecorrelated background and noise are 
uncorrelated with each other. It is proposed to separate 
the three diagonal lines from the background noise. Again,the 
same argument holds that this problem is a two-dimensional noise- 
cancelling problem in which no reference is available. It is 


further assumed that the correlated background is a band pass 


Moc ess for which the autocorrelation function is 


|m] |n| 


p n 
r V COS WN cos won 


Re, (m,n) ED 


where Oh Py represent horizontal and vertical direction cor- 


relation coefficients respectively. 
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Guano back to Figure (4-2) in the previous section, the delay 
2, +2 
1 2 


and diagonal line target. Then as a result of filtering, the 


d wii be sufficient to decorrelate the white noise 


System output (or the residual field) will be the desired sig- 
ñal. It should be noted that this residual signal is com- 
posed of an estimate of three diagonal lines and white 
noise as well as sme qanularity due to the fixed stepsizell|. 
The problem of enhancing the target diagonal line which 
is subjected to the noise (white noise and adaptation noise) 
is another problem of interest. It will not be considered 
in this work. 
For the purpose of simulation, the following values 
were used: 


1) De 0.96 Ww," Ww = 0.143 


h V 
23) MES 0.99 Ur S = (eps 
The variance of correlated background = 1.0 
White noise variance = 0.1 
Target diagonal line intensity = 1.8 


For the optimal filter design, using above values for Dem 


a a Ie Tener Hopf solution of Equation (3-30)is: 


E S3 
Wims E Bx Rxd 
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e m ° . = EIS , 
where Wims, RT Ro defined by equatiors (3-22), (3 ) 


e=29) respectively. 
Using p=3, qa = 3 in equation (3-20), 


R = 
XX 


lud 7 u«97989  Je9r:29 2.91989  c.96 19. 0.92138 ¿994029  v.92128 — ( ,BBA4*4 
—31.9 1989—— tk. 1--—————.91389———.$€-29———i, 7689 ——.95-19 —0.92138 —0.945329——6.92* 38—-| 
Je94)29_ PO E le). 9 19291983 Ws 29 6097583  Le76rN iS 1952138 | C4 $4226 
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Only the recursive simulation is performed and compared with 
the nonrecursive Wiener filter. The simulation results are 
shown on the following pages and indicate that although the 
optimal performance of the Wiener filter is not achieved, 


the adaptive recursive filter performs well. 


98 











e^ r9 S omen Go ae TF rr +0 e ^u rt. m9. + IPs 2 ere ET en em Ar Fee. nm turen. Sere Tu e MNP ann Fried 
1 1 i 

aa, ’2 199 DAD oe nnnn nu nun en NE REN RRR! 8 be 95 43 4 9 B. 79. 9. SN Ne ee 
wur: .279393 1272373 SS INN ul ge ta St t et P Pet S tt e ALAS NE PB FFE NS NR Uf Sot g 
nur 552 88 7 Br ee Se ee y a f, e te a HRS Sa fec t, e, o o e Te 
NOT oS. rn Lose O t A, Ra EA SSR SA Ff TER SA AS ASH, AT e fe a T T RN o Mt 
d)>):1.* 13. Ps FNS NSS NS NENA FRR RNS SNES SNR a e A A A SR NADAN a 
Sm Pan a- he tae PT 2. SNS) SRS NSN SNS SS SNS SS REE RS Se Ree re 
€) 9— — 2n) Bot 2 0: SRST SS WS SA eR Ne RAN PN SSSA SN. SR RN ANNIE IT TR AN NN AN NN 
Ge ttg a RR RR Go, oH, LH WEN MEN A o M A g T" . SNH DY RW PN SSN NS NN OLS TPT Fe 9829. 98, o9 9l 
PO a O o NS RNR NS Se Y A A 4 Tt eg TS cS OW TR e 6 ALS NAAA SL D $233; "P 
name 2 on nn nun S De PH tt N t oco) A ts a IUS y ee ty e US AULA ma a III In 
eo 08 OSS Oe PN NNN NNN NNN. add e] RSEN AAA y, Sa V ALAS S USA V IRAN AL tU ej ur D WI P. NO. CAS, Hs HR 
Pe Job... LAA AAA e ERA] ANA A A AAA MALA AAA A > NN S 
E uu. .1.(%.. son nn ann TA A A A A A A A A N fec, eR c e Rom, ENS OS BR Re Qe 
$- 478. 9 . 9 9 9$ 9 9 LAN N TITTEN AS T A STO Tre Pes wh NNNNA NNA er t 09 5 ee ee PR em zer. A a WRK KES A 
CINE Hoo 8 Oe en ONIN NNN FOF PS RN SRNR RNR ND DR ONS 000 UNNAR a e g 
wo * 399 99 99 9 e $ wwe Ye $3 93, 77 4 2 704.0, RE, OE ED RA He t He ru RT AS S A 
d "e 7. * * 3 * e * e. yo A O A A A Hte RN EN I TITTEN ee Ae ee ee S y PR 
Ee ee ee We, T w w?n wAn NSAN Na, o UNS tmd oma Pel a a a a o a a RH, AH A LA AAA a 
WA IO MI Mo 7 - 299197 :3 6 7 ov 9 tO es. EE DSS ee Ry P Ge Oe Nee et SU S e Ug Te T 
gue, Ph, SM OOH FS Se ye y e ~ ~ > de ds e eg de ds de de Do e is Dd de o tee” USE o a a e a A e A A A e g 


NS MS eee fec RES SS A a a a APA Pa AS O RI O A A e y y C.M UU, AH, HA, RH, ee 
CS EB. WE EN NINA INN NT NN NN UN BNE SN ES NR A A S PN 
".-— Ti tut DARREN ILI DD" e PD O ug HR, RN A, UO 
2 SAO . d oce eor NN e TR RA feos A SO TA A A FRA, A, S, T e A Oy 
Dd 0 eR Oe SR AN o o ii NN 
I es sd o de e o e RH Hg S FR, HA A, Re ee eg” e up E TI N SANAN aN w e ANNANS NN S S g 
2 SNM NSN NN Ne Se TON TRENNEN NAN A UA ew eas 
SS Ne ee 9 UNI ANNIE TU Se ON Oe A RS AT FR f 
Fe ee ee ee ee o a c o Hue) Auf PRA, Ra nn „er un NN A AA AAA AA A A HR PH, f, 
2S SS SSS SS SANS WN ey ~ So oN ON €, iy eg a a a 89, A A 
C49 se a o a o 0 a o a gg y NN + A NUT A A A A A A g 
ID III A RE ER A A A A y 
UNITS NS U TUN 
Se ee Ne ee a ee ge ee ee A a Te a Sh ae Ae Shs Sh Sd ha ag ig ag hy e Sag yg 3040799, f MT eS HX o m Ie Pai gy 9 
De at O o O o SE EN WENN GAN SAN M A9 0 ^ 
DO II I ee EE SS SRN RON UN NN RR CAO TNS WBS 
Fee G S o $ ayn S N S ee es Sag te hee Se ha hg y ae a a ry s, ua Se PEE ee ey 
Po A ss Ne he Ne u ae P, Rt, a ee ee The eg ee My ag Be ~~ ss 
SN o d db RE PAR ELE YT ee EN Ne un 
See ee OO Oy BN N UNI TUN TAN N Cw, m PS Re o eg m nung 
UN TUN NN TO AAA AAA A N U UI Y AA AAA o, me 
O t SANS ONT o UT II A AAA RS 2 39 7, 9€ T 
Lindenb bb EE NIT ORE a a wi: $0507. 2$ P tc H9 
^w o SO m Cw erae e SA S Oe Of CURSO S BRYN AHS ONAN OTT 7 Eu A] ~~" 
- 0 o t, ENT A NN NIS .. necp Qo. 98 
9» 9 ^47 07 RENE BANU Ae mee BS Ny 
c= UI SE OR OL ~ y a 
3 f^ tg TEE TE *o7 09 927.7. 9 
$207 7 * ee SOE Oe BT em wee 
Leld IO d e^ 9*5 7 y 9 9 om 
TS NANNINANNAN NNNUNN a N ~~ SSN BS 
thee BN NNN SN NN? SS E UI 
= . DA A A A O A AAA A A A A A A A 
PD Sec. a a a a Y o A YO IA Y UA ta) 9 ar. TE A AI NETTE A Sg 
oN Be ee eo Me 70) 989. 9 9 AU. AWAY T NNF NNS ENNAN Be ee ee ee ey 
u IT TE Dan as UI UT OR Te SR SR A A NAAA A a, E a 
Te RRS y 
ue Rat Ht P, m iH, Re, Pe, ^o ee Oe RS A ty ee i e o Pg Hg P, Hg jp 
ZIIRL UN UA UNTEN DRAN DIT BAN RU Ro f USA UN ARA OA, V UU QNT e m 
A A O A A A ee y a A a a A A Sa ee 
mu a SE SSN ey 
MT tatit HH NT a o a gl o a gg TU AN A ANA NAAA y 
Y UN e cé fe a O a A Ar ANAL IN 
~ DD 8 SESE CRSA WE WO CN ee NN. 8 CRANE 
fee PT I NN Se SN ee EE RN Ne BN DIN SAN NN N 
ss sd o do RM do 9 T Sow 99. 9 d 6 9 9 9$ 6 0 ww 9 NNNSN r 
U TUT EEE N NUN UT IN. Y 
w. ~ ee EN Ng ENE Se Ee Se WR N 9 oU S 
CoN ew EF y Cep 99,84 0848, uy, RR fu H8 ee Sig fo. 7) 794 948,99 Ee OOS Oe A e e 
A-— Tom mom omo homo P9090 IM ee A UT UN! SN SS EN SE a EN NETT N 
et O y TAI A A A A A A O 
nme mmt ^ mo UA A O ENT NN N 
i7. 4€ 9 9 9 9 0 99 6 9 5 0 EAN TANTE NN TI NN N NN sd P Pa Um tug p hl Cere e e e Te TUN TUT NT INT NT TUN N T c ug T S e c RA RU A g 


«4*4 


*e*690999106 9 999 9 "ut 
& $'*wepgte 90 Y 9 vum vt 4 


.: 
he 
"7.3 
..- 


> 
. 
a 





(* ^w. 







v7 ! Naar 
mer 


Festo treu ga dd r9 huie e Pv gp |: 
'eOP FSP HEH es" 


edpentt dt ecco sf te %ee gumweasnee es perms eh oo 
ee ee) 


vote pw were OO Tw Ole wae 


. 
» 
. 
» 
i 
. 
. 
. 
. 
. 
. 
. 


. .. 


$ dao aan 








........n eon 


1.. (eto do 
* 





Un” 
€ 9o 4. AV RÀ Sy a m 
ow, EE ee 
C9 oae et 0 9 


rd DO fae sd Peete we sew 


unten 





recono... 
do. qqoez=a 
edobro»s 


re ee NN == 0 LODO WNF 
2 7 SRN OF OL © Om Oe a eS 


~ UI m PR HP ee ~~~ = 


*9$090009P9949P9. 10000019000 449) M4 
FP EO Oe eee oe ee Od Owes OM mewo tupt" 


KÄTHE Ir HE EDIT qt 


Vu 


ergattern 
- -————m.--50$99-595000090992999* 4 


LPRA 9» 73,42 *.9 


29 

gue 
feist 
Ebbene 
vod 

ee 


* 

, 

. 

è 

, 

* 

. 
be eos feos 


dresor» poo. o" oo- 
F 
* 


LAMA tw 


-————À fm 
Le aqeage por 
d 

u-, 





* 


BOP eePehens Of f~ 0000 om——9$990—99099$ 
eoresdispesed £*** 69 mamma ewos os = 6 0HHMEDDD 
daodrodio ro JJ n motn 

U 

/ 

t 

? 

7 

? 

4 

4 

7 

! 

7 

? 

4 

t 

/ 

4 

“ 

* 

`- 

* 


borren pisr. (4/9. ^96 


soupord 





Ie gues 


dias 


9999-90 9.29 y Ne ee Ne e 0 ee a, RT Ro t W T UIID TEE cH, P, gg TR Sg mH BET e e A Seg 


e 
Immun I A 9 9 NN SN, STL DS SNS SS SS SS NIT RN AS, AV 
A ee’ 79 e et te ye ETUI ra EEE TC EN FÜR S u e mn. nt | 
^P Mu 8 DM A A f Ha TR DD 0 SSS SSS EN SRR EP NR 
» m. “ein: is A e O A o RR ii l t NNNSNN NN NA N N A a N y a A a A A S A U N N N N a S S a S S 
Pam OR A IIA A A o 9 Y Y NINE g Sa ag Sag ay he IST SZ 
e fr vielem Ree. Tag Hé Sh, ath, a FÜR, tf PH co fe, i Se ee 2 SSNS SS ee SCR fot to RR a NT 

wre FF ewe em ee LOO ow ug aHa T, Hs, s ag a Te g ig hg hg hy a A” BER Se EG ey eg Ee ee ee ef OR TO NT ee TR ee g 
verre Ta. .rogorao TH NN Be Saa Sae me FT 8,707708 709 itu oc eH e je 0 sy 0 o9 0 59 9 9 ute un 
IST et ge Tr 9. 7" ) 7 J ^ So, qs, Hy HS Puts y S. *3749 33 4. J 9 e wea t tt NT AS A S 
—-—pyvee.et engen... 7 f at, tt SNR RS t ma Br eV TITTEN Be ee ne pg eT eg ty he ty ee Oe 
SAS Zar... UI AT NNN EE WTF gg m, tt HW TD 
IT IE A A A ess "959 47 9 99. cw III EEE 
'3¥ reece dut ELE IS MITTE TU 4 7 v. 4 8-79. 7 945 S POE Y o Tr G P NNN OT enw DT... OT A PN 
le By r m 27 TU UT SUITE ETUI EN SIRO So O O 
¡reo . 190 us oge.7. Le / 171. Té e e e gg g D g 2 we SL TP NNN © ee enn. ST eee he ee en Teel dN 
EL gus I Oe ee IT TUI a uu E, fO S S Su Sa tma "e—- 7 d ce Q4 T0 7 7 4 8 Y 099^e^wc e 6 P e *u 4 e5455* $9 60060060900 2. 89 67 T T NT 
ı "et 9 BOP COTM Ee eID AT Fo «4 9 usu es a Ry p sott Stet I Pg “Ne eT UPR TET SC wr whe ONE. SECC eR COFFE OOF TT www 
rere? A ee he à 00 92v £s ee a ee a a "- 7 — me 9772.55? 595. e$ 979 4.45 FT eee IL Tod once T 229 9) , 4 
“Rhee, “ || OT EL CCA Td $6 9 7 4 PII Iu LI TVÓ SR T4AT* IA A E 2 Pe T 
^-.99, 13 a LETTER TUT LI NIT N TUN TUT LT IIND FD AUDIT IT ONE TE TE SE LT TEE Fam 
ir ll 41° CAIRO NATA TNT Y TFT OTAN IU Tra ITA nr 

1.Feos . TRETI pA R PEZE LL Da Si IIINNNISN Te eMteMA O AA 9 P A CN E SE Y e€99*906e690922329092 f 7 *.7 TY 
barfa: [sre ese e ett m Fy FF T ee eT ty FON NNN y O Ar e $2 2299 o 7 veo m po NN E 
A*°tFeeaeiea EN UT Y Y 72 00e T0 RT teta e unu hue a a SS Y Ty E SONNET TT ee SOOO ETT OF Fe ye 
pl A zga Ense XL Del mE dd RENCE ST EE ri E 
wg F arr Inu turn Te N nme Re Seen id] TT HHEESESH t Magn ug 9 Pita Ba 077 

- z 


99 


OT 


0. 9d" p, gg cos(0.143 m)cos(0.143n) 
1.0 


WHITE NOISE 


l.8 


variance 


ty 


* 


ds 


CORRELATED BACKGROUND + THREE STREAKS + 
zero mean 


R (m,n) 
Variance 


intens 


FIGURE R-7 
Background 


White noise 
Streaks 


| 




















Re A hen Pdo E0040. P ÓV ooo Pedo era tci ttt t m 
eb df thus sdvss chat eed cod Sebi fh oh ad sP etu lUl b st adl Phadadbecdaal fll itd. titi P. Fi Peds oth Ph 
Piberamefnad inle .(40. t. Pon it sftt do EE EEE dose od 2 ro 000 w.o, a Fran: Zr Zr Pin lad 
freee eed aru de Ld dee omer LES SA aa Od, FEES mer et ndd eilt rated IE iin 
eferoafflfofvttr/*e "nf 7e folo lol c dr DES PETE LER cf Int oft-mt ener ias B 7 4-414 P £o ^o 5 F4 m 
OL ef bne nl. 09 44 Bud lg 9 UN dodo omo dd A? LTE ¿db br ddr dlls do o dr dl do SP ested eo nino 
Self seno sala]. P4 «P vs do 879] Five a d (edis. o if FJ a. FL moneo nos] n ef A 4/98] P 6044. 10 11114ow 
**eboe/ 8904. S (0 09 4s, tt n £4 FF teho.. Lill tola. tlett adeo o fo PR E LEI IN 5) 064 S 
ware, 77b dad. PIE ded Sie ED wee ceed uolis d ta bh bed ce ered ds FF sam mol se 0.0 Lon font sue P4 
94 vns sE* IP uses 8f. I1] g 7g Y d- lodo! SENSE EA Ideroto o dd. "1d ds to P$, 4295 o9 ¿00 (am 
Males bbe horn, F49. Z FE Fate mide iol uo ¿Add A Pt uo esl PB os Pg eli. FA be ve Pieter IP Td 
ss. nf s smi P to fF PL 4» 0o 484 nm... E ASES IVA 8 Zee. edit Di tt f£ golf 499 [m 
P4PercP Emil so Io» cle rt ?erP diedro ioo P PP P6 tott, th as cl dro Pu os Pee Pees shade PS [981g 499 
"dd doa SS Pg. prc tit odee, LETT De OL doloto atile d obudi tolol o . efe coders o (10m 
td AO £v. e fefto del dd lb Pul P sg] gg. . 2 OP PAT APTA AH P unt hun 
FLT USERNADES E UT. de I? a os lie "EP as: I sp tt ecu As 
af ilef rah Ohare 2% ohome due. make fo uwdd ot id DO O e oa oe mE PILAR 3: 40505. x] ns fan 
eL anb LE Eedt a Pu i:t... rr tds. 2 : ced Pembsnndieee ©. Senedd, LATTE bs Bide 00d mat oes eaten 
Lledo Feit mmm Rt de ¿lo direa : Lif Odin ted sitli fesal KOT P9 Pb uw tm: 
Past salone. 0m Poo ttle TAHT E I IPIE ERTEILEN eI e ma S 
—^7/947 lll J7-0l*à—-2/7 17/0 4-4. P982 OTAN NS m ase Pme envase Tc CN 
4*4 P onn fo ( 2047.4 sfl e o sotfolofo.nn. dan. 04... 4... 4 «¿dit 149 Fo da Pr. 9*9 mat bhha/—] b n 
alle. LZasdiside Pl: 1 50. Li. ee ee 01141 41 ] ten t n er ce c9 -me 
Pie Lod Panevsddo asi oda ies e. ando vater). in e, e .aiélrtl 1. dla. d Brod FE) of «Fe (4999 nu ens 0» P -h n 
BP ellaseramı dianed soso FOtol Forse? Josho.» £ Fthtnt Shoal io FE De 4 PIE warme Pos. c tt v IN 
44. sd ot netu sP 4 ndi, 8o nt P 46g Fo Parc td Vb OTe wh hhh bed rr oae. ddr nr do di dnd llas 
ETD 9e tar tfe Pg. [949 HH NP 41 VOI. *$- oos Saale vost os PERI alte 9, Poll rtl rtt ena 
«*] ano-hora ooo trado rd ¿90d . 90d ido. dt do dr. d uod t rede. te } Sal efc d ocio anh 
Ene pnnltflo^ (ini-oíre^ríns-*oe-9FE P ull ol Pg hr Eo i Srt t. ant rA [rh Io lg och Boo) rip ^pecnAPpto tH 
LET cd FeO OO ent vere Oe hohe OFFER o, dos os ffo. EL Pdl go mud tod PMEPL 7d Og. eo hs RIRS Smet ir 
rm raro doo . o Of 1d ma covers ora fie hor fh WM enced cok hinds Of: ont sd. FFL. a LEL Utosjoen 
afafanue? o l veh -lrk » Pg de dd to dad III. 1 «In... fre Pro PPP Bg 9r Hot (lg eon 
USED edles etant Fes S t ttg 7 A AENA ¿rd dor UA a HA S no 
Bsta nennt doge Dolo. dos d. e/o .*. 1744. 11d LEICHE EST LE ud. Oe door ILLA E RATA Fo 
Sellsanaisl sl lanasiei Lot ds ll AA AA nave solo. uo? Jo! OH Ud on rn ZUR rer tn 
ansast? doel. ao A IN andar bas .-7 7 derer. nd ed CATAS ESE yu Ur ttt. ta Fe 
LU ee A a AA A A of e Ae LEVIUS TIR LES c Sedem den 
Q^ 4046 ds fdo e 74.9.2 Foll Je sre hd. G0 Pde PEASE ET > ortt. $. e£ Pto t ln f'on 
eT P gBjPLIL8 £0 98 t T It PI gt 4698 |] |g0 TES Pelle-@.. 04 1 dat Ve HI uti fe exs: o 7h 
toloj dlse? e joe 8 iP o MEE Pug ag eus SP Fe "ET E .... 44.1: Pg 98g. thee 2 AE F270. 7 Trew 
mn oo lll UMATA mtd fo ocosthev. P ns oo. L4 und. dinot BLA AA RAR 7 04 Ll e fh at Cen lank ad 
dass hid sh sb sherdbinds +. ino sola F. atori. d de Fi messy Atela e.as.. Foltok. VIE 16111441 lll 00m 
LATE 90802 bed 0/0 Irland innen d ol l6 AP natito ¡too ddr sd dd A | gg. tarre fehma 
tarfar yai fat. sel EFi trat JE PER: Hilaro mele tin stet Polls ./77 »/> "P ARN usua cop od 
=æ sat} laalia] iail odrat -asilai t tilin mmr dreds ced uesdfenanaabelfrefme—f sr l-Fh- Den unten iu 
29s s. d 89 infe oe P DU E OP gn ond e Plone of onssn Zell I. P istodi lodo $0.44. Jot fet 7 Jem 
bed us E n^n o. 17 d moo Pg lg el ua P d uoto usu o. Fod. F mh ct ht Oe 1101-34 P ve LH Sr iR d 
^^ qur heut .... peor off of. om e,ro Jeroen Pid lilis’: P 171? oh slid Pe of e otc tel 
POM ode dudes 29 fr otis P f. PEE B 40] tius oss oto Ptr] oS P 1 d +. 411-111 111191 Il 
HH lehrer PP tidie. tbt tiles fo dtl pulo Ps r4 M 4 | 0 hCN 291441 4.4480 IL 11M $ Ls 
ef, os dle P491 £444 n P9 eier scam PLI gMelfeulffl t V IP i FP Ogg |. Tel Ion 
(sine PIT Narbo. te dicate? ree -t soss SOS AL on? tł ud o de Jv. 8 0004900 n n^ 
ili? edis scent Pts Po tn s£..8*., P254 I IE ef 04. D T OCTO y) „ve outer forse „ıllre ani} {wr 
alles. d SEPA atjar TIEA ll. 44 eR For no nn dl Confor. ll. ie? IT tem 
seslene nl *E do E. L4 £08.00 go dlle- fAioli.. natoto o df bot. ME Pg gg 7 34%. .o. oo». of Terre Tr. Iren 
“letter Ran aaflet Di nd EPIIT S PITE PT PEPE MP LE PEP EL MEME PENNE PD D ERE dde. ceva teed ona 
4l nli P ed Alu to te. da s la Li Endet. I I votre. tin Paste he. ¿dora 
Ph dl ca hot Afs 0 Le. e. CP 0 d. 00 B.E te nosnfl ob 450g] |: Vu P 8g | oon -d dol din da 
and If eilio litai b-fit urn le andern. st rat wie pufoo.. 04 PEEL lr 
ati... 11 4 .£l dt.» PERE dl ¿de or ld ll lodos ld F Fe 44-9 [tt fo £À 1 0 4 028019 5 
aleman lll d ler d lidad dl bno bo bol S IRT aAA ATES 001] =¿ab=-J-2a0- 141 Peotidtllire 
ale dtf tul Rer. att Po. fa eK dos A eee un ddl “47-2 otoa „len C2eamits inn 
ae sil lena. SE S. TE Erbes tel... |. Bee. ve whee ud. 8.7 tot] fesd} fioo ., 8. 1... 2,0. /0m 
II. Erbe Elle Band Iona 9/59, sd. DREI Messer + . $. off osiop osef??? t i?on 
AO EN lont.. Jeo PEP obo. dtd 4. auto. ly, dt. these Ted ¿11111 Ita. ha: —....0.0 026.74 
Oo tree Pd odo o ed 1d11da 1944 Pv» on Paises. PDT endtonndun dolon ¿ddr rr Lo po dr o. 0 0 6 ul 
estf. c^t aont es lf n Did lo adan noe.o / shi teanshe 4. Fed. 444 9t - edo 4) CEN FS SOM estat Jan fa 
Pel b esrnd deoa o o. Deere DDP EIER 8 sl so uU o Ponto b idt Gooden fads whe nd P o9 t lan 
—afi^ef /na*n«í nmi nísf [ife Pl aor - me -e . fafs} sas uA nip, foe /lgli-4- ff enano yr emp 
48 0... nas.) td. 44 oso 448 lvenl- Hd... dl 1.4.9. 0.290) ..4. lad +: sèso t. n es. oae rt p ith 
«74/4091 46 S Pn L1 1d cts £0 20440 ST 41g4 4. AtA? Pets dom Lohn 4 P8609 daweg FF 2471-9 tislima 
untl. ists 4d 4d xd A 4:9 OF m 4. at 3 1dd e dedo ro. so foo. fo u sé fo [aldo B Leer Prim 
»ndatd mado re. nd xo dd Ea 2% 4 list. EIL gg 045b uos ut .v. dede Jo. Id. Jete. A See tert 
af ek BAT eden sed dra 3h xr EL ELEME NETT UCET + 4 311.0 114 11 Vedmarime 
Lo Joola o Jo... Exa.» of 14313 o. 119 1414-14 bd. che hie tse won... dado 114 87 144 t }, y. Fel -* nb 
sl .rdleidi. o dug 48g 31. VBDALSEDDCgG T1 S2 EE eer A > ezopo? Pío ^49 / Di IgA ht / en 
na alla of of. o I a Ir eos 
dlltetatrs .J.f cede eters nf óÀ 05/8 PELLE 04 0 4410 4 ^ De Pl dodo ra > -è of ¿opor fdo. 104. 0/1 
dbiot. tet Tei Te hoods „wo. ttot? GIA 4005495 40.8 MES Li 9 SP eect eines sé eld Lebar el Ona 
ae A ee at ef. Pg fs... 3H 72. £ (99 «o ll. 0x0 frorogodo. ol. F se s4 F P fos Pos. foh 
La hm loess whl wh FIAS, wtr o fatit rd... VSO sete 0 .of. Pippi ieee oo oP - cof t bot t rh. 
4 do Mf ot Is soob. dtes IEF m FE OI go AE, EAT gal esum PR oo ES P iisti, t "wo. e? 1t 91 gl 6m 
dd sed .donmarvadomnmodod. 1390 co dd JE > 1333-24 dde del 11d Ft ner lerne De AA DF Doe cheatin 
Dover ona dre de] e FÉ dE MESS FL. P dul t4 n P than ms nf. costo - y Jo F9599 068860099? nm 
molfenllilnaf eo. eos "le nl. l of fans eed sfar: -theadd loot moll ln lal oí ea sao "ao poc/fíbf/st**sio em in 
ed LP LlB go Pte Mt g95 FEET he TEA AA ANI AS AO do a por 4-74 s4 2 Le Fp Ii pr Id Ae fon 
Prfomolalroono] .d 100. 04d $7874 hoe Ele | domu) uo tnt 404. Z.F c]! AN einen 
ATAR EAS SAA ETA AT TE TAT Er de tut bl Po ius 
a*Plselfst alos dut e.e. yu P4 44b... HB 4g 9. « behind stios L0 gg uf He f, ffo Een PET em 
doled F4 nl g 9B eats l0 Ft (£2 sth] Pu LETT AOS Pers 1ddd (4S8 fro. do cp ð e epe 
alis astiti. d o tE us ost fo P4 4L P954 ns] de . Oh ave pee fess do Heiden osfosjoj-arfio eh. nur] Tann 
PP. + dl dde dr dr œ, jei. ofeoes, Oa. Oe , Loanne adonat yina. j ewt d 4»)? efee Jodi ate 0604 =$ Jo nm 
ala a aa a asa aaa RT fg g95 45008 In dor) oo comas Y q — ed 
por. bzinatos Pron fa.[/ ,n. Jar .raarojom -0......t,.0O00f000f ff 900 Hos4a 4d 809 n a EACH me Ow 060% 4pPh 
Lemna cd mlod bee had . bree FO attest mes OO 1mmt= 00.220 ,Í v. o Jo yl soo Ofer Os "II 


edo] fotit aftoe fF n af eof aum voo eton, pofi o? 1/6 oso ru Poe RE o£ Er los = dei tt ch 
Potlalalliarolirtialií.d Pde ce =fom. 1 -, =. 1.0 ,= 0000800 ,, of, dot oos dose ooo (4/7 f[/ 9 voee 9 ozon a} -an 
ett. dido dE dodo]. or. ..eo01rr0900 1.00 O462=4 20000 ee IE e... foo m. 0 Se cmt cabra 
AAA A ATT A Re AN] eo mn. Prater nt AT Hl hh 
BORER AMADA TATAM ann RAT RR U DR er Aa au amt | "9944-99, -7* nep tire 9. o eb es d rom ^y od 
mad MAL add ented td ^9) e^o | wdd Gabe | ndd trei l HR uo.) mre] arras do rf a o 4] an 09 7] 


White Noise + Streaks 


100 

















A SE San SB Enn Sn EN a a As e a ds 
g 79 2 "9.2 so Heu eNO Re Sees Fy ee te eo ee ES ES PPS PART PPD we wa HUNAN. THe NUN DWE ON 
dae ds o tS, 99 ds a9 70009, 9 UT ty ss is 98 70 Ives P2 y 569 2 X BT C ye MES ee ee eg TN! +.) 
UT IZ NN ON ON 90 0 SP Pee tuse AUR IVA Y IL TE 2 5 6: Aa ANNAN e NRI 
II IER MPa S e edis PIRE EUN NV MESES Su Nes SS NEUVE E AO A a ie mum ma Se Sa MR. AE RUNE Su di ees 
Tnne Teejo * 9 9 9 9 ) 9 * T « o OL © RE ee Be G R lg O y gg g g a DA BL) oT nn un nn non nt es ds 
innn; o... LOU Ii y A A e 92 TR PO A 9 944 9 7 799) I Ow wal a WW N 0 y 
EIA Oe PRADO AT O A FRR NEN TNL TN DT TNO AI NN NE Nne Nnna 
eer ee 8 ee eek ek oe Fo ae FENN) SNE NNO NNN NR NNN SES AU AUN a a T7979 AP IIA AAA 
- "97 $4068 E a $09.47. 9 «ow. Sed ee oO, mu) TE o Het, RP A Hi FN Ne oH HA, As ee 8 > Oe 
a Ne ee tg ee A NND EN NN AN AN ED END ISIN "ET N NN ZT N ee es NO 
ANP Fe NNO ew CNW! FN He OD Me nn, un so as ss TE I NT INN I 8 
aS SNS RS Se II LO UNNA A A AA AA AA Te e te Ie TE ee 
SSL Se NT NO NE ee TS Te Cay UT foo eT FO Pe eg gO SE INN 
ENN TN NOS SNE NAMES A DA gs fa em eS me eg NT RN ENT NL ee 2 PEP MNS PN NT 
PS A A NA e A a RNIT N I TITTEN 
Y A O O ce Pe fue e O a A AAN NUN NUN NN NN N. aNs. NN SS ON 
NN TN SET ee UT IT 07 929 7 0 et ONO FT NNN II e be 7 mw 0". ms ua A Oe 
PO 9 9 9. he 9.99 9. 99 US SU" e €* f» gy AAA = fee e e TOT TUT NN NT NENNT IO II TRA, LO Re 
de 479 9 9 8 T 9 09 ow uS TOT FT IN A a OF Oe N N N TI 
AAA z AR a mm nn. ALTE AUT NT N I -- i eN A rl UR RR a e. 
As TEEN 5m mu ZUSTAND IND NN DIT ET AND TEN NATION 
AA M0 00005000 DT ET NUN A TUT A IG LL IN a AAA A? Aut 
-—- "9 e. . ERN U o A AAA Mg = * ~ Oe ag ee Se e A 9 "-— "9 
MT O WANES BUA TWANG a ON a muy. ees SD ENE EE LL eT Ge ee ee HS FB ay 
LEE VE. SBR HET. NT TEL En ww e BE uU A. SUN 0000 77-9 7 ev AW oo 9l. muy ET U I ar nn $9 Wm o me 
DA ss ds a o NUN NN NN NT Poe uS t N ~- . et — nn un 0. 8 9 9. 9 9. 909. Qo 9 S ER, = Meg ML eM ete A 
e. . c - = . 7-9 LITT NINA NUN NN NN w aeth NANNAN. NANN YNANA Y. e, HAS Ius do u 
tane .-...... = Fri TUT A AM a ~~ Ne = , 7o9. 0 0. a UN UNE O o NN 
nm. ee Te ee SS ON ENE ON Nl AAA AA O AMA AA A : .. = U UT UN NN 
ST NNT SSO DP OT Oe QI e Je Wwe RS NS NN ST A TR a N ON An A 
Un 1 Tun "E . o o AI g AT A e ss RL SSSA U RAN HE TE oN 
ur Tr TI TUT TU I TTS eA A HH A A Se oe a NN TEN AN NANNTEN We Owe PL 
PE. NP eg 9 9 9 eom, 99M, RR Pe eo I IR Tu UT A Uu A A AM A AA 2. 5," B. V wow 9o o 
Lune nunaa SNo AN TITTEN NENNT TUN mn. um ne [~~ NAG cc 
A ss ss A IR RR PRU PR MO Tomtom TT "08 9 tua au NH ee 9 8980070) 9 TUS AA A A MM e a A e Y ty 
fo 0 o Meee o A A A 08 is ss Te Ten XT e NAH TANN SH TE ee, te yo A-0009 mo. no mU S 
nm ti. ooo o. IAS AAA as o adas a SN fecu. RH $T Cg 
^w uw*e*elegeRt*.*w- TEN A, O e O TEN HM U ee ee ee TO Om te um ^ 7 7. B tu wwe 
ADE ANA ° -N NaS we FNL NN RN e 
7.9. 9 077 095 9 09 oe, TRAE RR SB TAN > c NAT UM AMA IS $9 0 © "eM A A T UY 
V€-cWw 9 $ 9 9$ 99 9», 5 ww NO t m IL e 9 2 990 NS RN ET Ne OG eT eT fe YNT S SI TUT O o a gg a e Y 
Lo 9 4 "Q2 A AA TL OHM 8 yr As TA A] un a, TEL LT U N NN NUTZ 
mn!" 7097 095. 457 - M; 9 MALA 8 BNE! ONAN OW nt nun. n O O a o e a y a O gg HL! a ve fF Tm = SBN AS FE NDF 
Senet. HDL TAT mm RRR ee NO Lib p o- 3 AI a a A AA ss 
Ass TR A O O nn DEI SUN TUT wy 
PNF airy‘: Berry m, TANTE EN eee Ne N AS RAV O nn jS 170 NNa ANID, ANANN D 
at? ..o ndo... As o a do o o A VW ey, a US, Ne wT BLO yQ 
wie ee .. 1 A AA u "7e 9 2 9 A e 99 AAA A A AAA JOY Ys Cm, EN Ti es PM" TF ON wT OF OF 
-—— CoP Ete de Vom or mund na BC EL BE ENT ew EN "-— Noco goyr? *oy * 9C ss A TE RSS m, E 
"20060999, *9"—9 NN CMO fC 9$ 9 9 9 9 e 9 9 o o CVV, WO WAT 99997 -7 a) et ot 9. 4 ew PON NN TA e NE 
Tw-* 9T c4» g**3*—-7--—57 9-74 o9 9^ 4 eS BU. rm nr —2459* B9$95 a9. 94,5 9 e A FT Sy UR, e Re UR NR, f 
nn ideis cec aee Ree Re SE EEE A e o tl y A SN SNS ec QR IET 
SN Pt iesse, J) al Nonan pn enna Z mnum PR 33. IS IF RI IE eM, EN nel N, AN 
10m.” .» 1/4 SS A RO O O O O A USO A O O A O E E E te paa 
e FT Oe Oe $2393 eT SCALIS SR Ss foe RN A A RN ER TITTEN DT RUN N NN AN TAN. 
FEN NOT TOSS FT aT aN, MOS TNE ND Se NS] See Ne Ns O II A UNI e o TA RV UNTEN 
POTASA DD 2.4. I, M. 089 Ce», te, RI e gy e HR RR UT NN Se EELS EN. QI CL DSN RO 
oh FOO oO OH at NO FT eR me | A AAA Y y doe e tH HN HS HR HR BN RAN NDS UN INN INES A AR 
wm. Be OH eM Ode Om O EHNA QT Sm ON OO A WV ALAS AURA 987 4 9 t n T ao c 
"— an *€ - $9 ,. "^ 9 nun. Ue o a Ne ~~™ <a m nt. RAR fm Pe ON Hn “mT Fes SL a 
"20—-e9**N: B5B et 979 49 A7 * 72 SN OA DEL Ie i SY MBN NN OR Ol 9 909 S8 ALS RUM T ra T S TT 
un mn. dena toot. =. è N.S NN., A A A A IA A AA SRL 9 Me AA O AAA A AAA A AAA A AA 
~~~ ee -- 7 8 9-7 9 9 487 HU) FIT M DET un. m Inn AA AAA A AA AMAIA ONO eV UH, a Y ee 
fc. o0» e AAA ow £9. x $9 97 208. 703 tae a ON LOO te Sw ho Y OU UA O NAN SK UN Teano 
A A A S aa ae a aa a a 9 NR AE Re RAE ie» £ 
FW Kann. nn one $7 WET UST TUE UI IN EL ET U eNO OO ee OL OO c "e oj 4-9 ESN FG YF 
T? 7€. »o- BEUTE BEN ET eo 9 .»._... 1. .n.3:% I: IT e O A TUN EN DEUTET PU UT y g 
*"ce ^om em eo pu wWuvawW a9 7-998 o ow foU ow XO He Poco s S o OF, Un NAT AN o9 ST y 
29-799 4$ 954 a $ ^" * o 9 9 "0 e A 7 97 * ss a og 0 aeo A 9o o0 TE TH EL ey” OD 
mw v9 M aV, A "uu eftt Yt MA ATINA eL e (wo Un UT EN U e RU ea 
mr mt DIE RENT SED SINN DN Ww en mp Nm jad o o o e e TR AR SNS SN ew O AAA A AA as 
AAA nn E RA A A Ow UT ET NUT ET A A AAA LISA AA AAA OCEAN ~~” men 
ss sE NN il un "9 779 * $ ow" NEN NNO OK ee! “eee EN Ow BN WE PW UI TUI e 
37979. 6 * 9 9 0. e 9 9 USA 9 9 pow * wm, SE EN eR OTD. Aw IS UT TUI N OT I nnna y 0 9. 
A A OT OWNER AY TNS we = ANTANN a a UA 8 79 9 6 N N ~ sent Az 
$7979. $ CR oW cm. a oam e UR, A A A NN * 7S, wo 07 apu IS -o s 8 8 4 P8 wc 8 QU. gH, He, aS, og Me, Me Ne ee te BO A ENNYN iD 
mm 2999, A AAA CA AA ARI AA "e P o9. dU. 9*0 A A A A eR SRS T a, ^-—— VA 
ew NN FANN TN SR T Mid de do ee Oe TUT Nt eT TT LT TE TE Oe Oe HA NEL S9 werd 
Nae RETTET TOT LEN AN TI NN een » foc. wc. NNN suc acu 9W $09 9 79 $7 AA e$ 9 MNT AO Tw TT 
MMT Te 6 * 9 er VT co MM ot exu a T R o o id TOP RU m "ew 709 9. 7 m 
$^ e. 99 ox OT A UT u AULA SO FENN FT ee ON NN Ü mu (0703 l8 09 T eto, N EN SN Ns 
$9 7» Q ^90. 9 e m » 9 A A A A 99». SHOT Te UN es -Z tag tT SFT RF TNR RNR. Qe NG 
—-— e ALNAN un . a A A 9 eom UI m mn.” ea a * ~ 1 bean a Ne TUN NS OP NN 0v —-— m 
BITTE TEL TEL TUN TI Me FT Ne BN TN NEN (Sow Voc TNO Ot "-9 7.9 87.9 RE 
IE TER Inn dus tU NY o t e AO, NNT BT NF Oe TR TN RNA ON UTR 877297949490 9029 e. T. UU 1 
T 7 Ww TEN ET TEL OLE ON wee” PO TUN T T.F OI A RU rr Ara Y 
POF SOT eu MESES L WNT Y ee OT ce a sd e 09 49. 9 f mue eu) € FELL TS none nen INN U 
IAEA ss ON uU SR TR N AL 9 $. 9. 9 THE 4d." . T. «o. 1 Cea € 70 hoe 9 V e SW ww T 
"e mn TATEN Ge OU Oem UN LM S "e e a NGANTI ANN 9. 4 90 07 WU) T Ot OF  E RU .)15 
"^e O IE E UU NNerNjeMWem) c NeeMAS es f eS ef S Nu NI ser ERES 8 
binh E ELE LES OTAN NN 9 . A oN UT TERN TE TERN ASA 0 497 777 087 9 0706437 9 a.a ANM Fr 
~~ ur to gro nme 7 9, 9 CWO WE” > ar RR SETS HEERES SS ISIS IST TE ET SET mern mr) 
e NA A A e o A ao arado as MAI IA AAA 
tunt Ton OF Re TPN PMO OT EMO OTT Tg OAT OS UE NEN OTT Z eN ONF 6979969 4 454.17 7490592 $7 T e 54a L0: 94s SNP 
eo SAID IAEA LOL. Et UNITS AND A A we" eee satorzizeralo uo. y 
me » v0 9335565697 * 9 O A my 9m T SV 7 e 7 9 V UI 235 (eie etat Rr 
REIS PIE sonne ran nu TI IS © PENN SN OURS TREE. ETN ON NOH ENN Weehee eee beTore el neers 
oo eae mo ade eee eee eee eee BB NS SN DER SE NR ee SN NS Y ete y m ee Ag S me fm T ce Ne Na Ne ee me ie 


O n N S ANR NNNSN SN SNNN SS SSS NNSS SSS SSS SS SSS SITI AC 
! 4 


Noise 


RESIDUAL SIGNAL AFTER RECURSIVE FILTERING 
TOL 


White Noise + Streaks + Adaptation 


LUGURE R-9 























-— y ess Ys va y. wet? 
123420 ]34-1. 3-3u/& Y «2 62638 1. 348* ?»Q  CY&ELY 
ee OT T= 
wu 4n4548090 09 44 09140077/ £722 / 7 8 o of le 
nm nor dt Hi ss ssifoarovrieovserefs sigs 
RAS ass pd rro ddr so rs 
AA 001 amorosas rr 11 
Alpoormarrr mo rr 14 e [100 dro redes o rr dd tvs 
MPO mn au... $009 11 Poordd¿ loro ld dro 
pro =— ===, o o5 )E AP do/roróti to] 190 
PARES -anvor '/orrdbilora rra] o 
rr --—.e4.5/0**b 40 ooogoooriape/ /F ve 
AM Pm war wees :i/»t2bbleabotigostehpeof iv ve 
ru wee eee foovidtoadrddtenar os! ie 
Ar. -———.9 1 fo7à&bbtóra44db dioi? /7 9 
Morir —. —ocotettot P) visiAbdamoereocibheo d [0 "vo 
?»Mot em——m—-—- si^ ff''ecobbbibadalaspeet/l ro 
39999* oooocooteoccottt t fll soos 00 LO 0l 10000/// e 
Mel 4002090000009 909« 1 At 13/2 /7 o5 499*5a92/// 
PpMAd«4R6 os o0290920' (^ 1: f£ B7 (ooo voa tes Lil 
mern iiv (g/l gom ioo o PP 471g 
MM«4io »oeo90000o**441: 4 lllllda roo oo 1114181 
ee A PARE SELLE ELLE LEN oer 
et SEAL LEHE OEE ECTELEES UES 
aed ive ye eet bee sett Lee: 22 € ie: 
meee dl ll rl ll llddllr oo sus. ee Ns es OF 
"tttltttttlbettdt llt tttm viene pu eum terne JJ 
MLL LLLP LEC LALEA LIL ELLe . e v dd 
LLAMAR Hutt 4L L12 01 Lg (14 0000 (Pm Ut was eee o 1 Fy 
ilitlitor oro lL FLL ELIA stt tt tn v 
MEINE eie Se AMIRIT T E E EES enn: dh 
LEELFecococrcvevoes eves sd fi su rtH od pvemaamsove se? fe 
dilltorooonrorncror lidad 1.0... marrano dr o v)? 
ÉL ltlsoosoooveaocel/li(* i^ *e««ac—---—o40se 9 jf 
(LLL 0e» o oos opea // |] Irre ern tr ee ff 
Atttiot sions ooootcvol/ /[/ s 1 «o0 «——9—20090911v tJ 
dilltionroprrnrn modi 1 tnter o npe 
(CULO EL et ton n Lg 4L 0L 000 9 0 oio tte attt trt JE 
K[4LL [4 oto oho oov oa P ([/ [1*9 rororo run 
ItLA4LCL 004 LLL 4 1 040404 LC Sav tette YS tv; 
CUUUratttttulb ALL LILLE Lite es AD IT Te 
ttti 19 M dace yp 
£ZCULLULLSn gn g0gL41n421000Lq012 77a ne Merida 
CLLAALELLE LILI TLELLE Oey ate e N 
HO) LL LUULEULUL ELE EL ELLE 0 0 1 1s tp ewe error ons py 
wt ZIEL (o 0 oot. mdemot we tp prove 
HM AS AR An o ® IE 
meade g inut 05415,2949 AEA SLL LIE Tite 
AA A 4 weta wn n AAAA” 
4494994" aquas 34 ZEZEAT LE LIE LI ET ER LI 8 
are HI DELL erw e el Pi EI I em 
144420000099 F'i4 1 14 Faded sLvervevess les ees 
aM t«4199990941 10 S9 131159 FPLgLg P Lt] rà ggg. - 
puq44944^7o090001411*.10,. 7724/44 t7] P074] vs 
44440909909 909099*91! ^" A£4445644 0204.7 7///-^ 
AA eee omewentagqe-—-Lf/ffjlesrserpoereseeds si fre 
1b ase ee ee ee Cee 1 LELLEL eerececvesaofs ss ys 
MM drteewocoepeseouvaqet sf/Jisrvavqseeoteoesss/ i fer 
*Mjéoob ' dor rro ooo oo o] o 
we (q4Ho.o equ erevomitis /¿d[[oornnorrmtors ¿o .v 
Pa4]weceboo voce t^ | fddddo morral ld? 
vMeO199565o090s09a2*9* [44 /f/l//*st oos otto 7//[1" 
mn 4494990? 0pb l'uso £dalduiíutootsootdoc/7///* 
man riti uhi ada lldiltilaeelipit tl eo. 
9493 4198 986. 169 tt voe PRAIEL T4 L 2628 P9 P PETI P [190 n8 
PA Ar Atwaa Aiea, FELT AEE ELT liters 
PODA TI IRA ELLA LAA LAIA AA IAE NO 
AE ELA AAA CERTAIN AA N 
«esa 02 ELL LLL Ll tiLitlorttot O UE IS 
CALLA LILA IE AE ALLA LA AD 0 e € mt mmn BET 
CLOUCGCUCÓN LL Un 1 109 TL L2 0L 1L L0 6 1 cip tot w al 
aa cea crio OPD 
. (CUbttssessorvoe/4/P 4 (040. t 1r moore ro mpg 
£L S EEE AA DAA V rte th i t pup 
4£UULLL 05 nfl 2g 15g 46 16 00405775 t |n gomma 
tCUUULLvvsosootL (002417 [00 tirs tn jp) 
ddllteocvcotilillilill > AS10016) 0101910 O18 Lore Lu 
LELL1 AeA eves hi ltititiir er AUD ISO f 
HCGCUUnLnDn vv t ebhu P4007 (4 P1 9 v o reto toria! tcm 
a re 
OSA ALELLA AAA 0865 08 (00440 0t omm wo BER 
BARI LIA AREA AAA A LAR titt nr dl, 
MA qa pte pe awe "ELLA AS 110941 of peo Bertie 
URLS TIER OT DS TI A AG ee ere a ze 
MN a ad Ad ld ll lt. ptt 
MU quam da SON ZAI} itt | over 
mmo saaa 184 JPrikdllbidHJIgqeld dà 248g (9999 
"X450 90uu0ut42—- 4. 54—^////PPlo9 ge g/ PL / 2 4174/45 1 Ho 
4440400999290 4418 1512472722272 RD GG TEE TED EIS et 
wm 454 100009917 tld 420422770 4/01 7088244 10140040108 99 
mri AIM 115201017 727777722 4 04 L2 4/1141 0L 0L E999 
«$4444 teo b 900445. * i See FETAL LIA 
MUI PODIA tt //77//f4 v*»v9o*v//71//* 
ao HH in 4A P4900 t10s2aJ A2, 7t v 
m4 4440000004. Ltt ! ltd mmm... P P) 
A496 0909909 44e 4-— /Jf / ** b otbooo/;//7/o'v 
A-4459 9999?59899 71771 NEPEZEREEXEEZEEINEEEAGE 
YAAA Datoerne amr t SEIDEL ID Sen er EIER 
ww444wioso 495 0)109 2 1 227277724 9 77 PP P 9199 
nannte a 19 co ZAZA it titititi 
nme ai Ve REIT REITER DIDI LE or 
TEEN IA ortti’ 






















































40 |? YA *^ RO ^ Q3«*« [45 ,274 5c [5^ * *14*4t 929 *?365A]8 € 
PU Fe pao a PERAL RRA AA AA RA nq i 
NI IAE EEES 2 
t tesooccoooceooeee' 1i — Z?/L20Lg440 9/00 P 0g gll 3 
sao... "I* Jf gl Vililf 5 vtov of?) [f )' v.c ^ 
e-.eso0i/*' "J/// vr... 0.0000 00:74 ""oooo»» > 
y 4 o — 00 vr Dj Jh Io Ho Hohen at / ss tateovseoos é- 
t. - .. eniro/[J¿rerr ql lor rm. ///4*o0r000.. F 
$- —-———»: 9fjgbÓrvevedgdiblhteocss/7//"*506 ——— K 
er ames oases Perec Push best ovoeess' lo peneene $ 
eo. womens LJfffreordasEh sb posooossy tose: -2 s 
Pus ev eoe" PLaesoowo deh bbiteeevefs vo rsonano 
oe =oæ=oo't't P: glllosooctbttyàbeoooc//['*" a8 -—— 
om —-——— oes fjillleveeseoebabaebeosol/]7*osoo— 1 
eaman. armo, 19 I[[ 0 oh o.9.90II1J 9009 Mmm dy 
t” eo. woes: Sif feseoetscosreeots sss Voveceem § 
9e nm 001101! [// I! [remote oo oJ | I iNYro os, 00 b 
"eeoeos*40oo—c*9ocsvoo "" iPePg gg lC L9 is vot) ll] 1 tnr] ] 
s0080- "OU CIL ELA AL 2 d 02g 0] 204] t 9 - B 
Soo osL LE LIL LEELA moa g 
o erea AAEM aa s 
“Han A ver eats (| AIMAR AMELIA dre sm i 
Vaart 1 eet TPES SEE LLL ELLE ELE ES o2 t sS AT an ao 
ARN OUE EAN ESE TEA EEES AS eb ese eee cence ei fit fe y 
EULER EE EEE ARIES AN E NAAA LS 
CERLLELEL ELLE EEL ELELTL LEN VO 19 es ome cece rei tite S 
FILET Cece mF PL LEE LLL LELEL: UVES VEOH Be een SP UP TUE $ 
Pitorororarnnnn rol JEJE LES hmmm tee att "PPLE EE 
[foo sarnnoaj dl un Atos rest Sie, 9 
(995bbbbáàbeteooocc//p ^ Ct were oem rt Pl] GS 
tornodbriditzbtaierormoo ¿7d oe „..........:® LE 60104 — 
orarddtertddrr ero [fiero % ceevesseoess’ ff to... 1 
fered bad tseoaos eoill * ewe losossaeceseotu vf bre... ¿ 
4*9 bhdiabtdbeocovof [ [o nva eov 000*"v' 77 * o» 09 ? 
feoobigadtsseoooressse 079: -—iise ee ootoco ess sonast 4 
RE BE DER DRS ISIN I ENT AAA TAR $ 
(/59e ob ots on aA P] co^ tiov occcocve'Ut o vrs... 4 
pete eee ee teeter ee ee AO 7 
Ilo. ..oovoernrr codi? : Nenta. vI1111t1k A- 
filie mee eve etf dl doo IRRE sS 
L0: t [EB JE TDG LER MIN PARED 
PEERET LEIDER GULLI IE 9 ~ LEE N 
RITA RARA RARA AEB ES RA AA Urn aT € 
dlllltilitatilrrta ohms . ALEA ee 
STA T T A T O a T E T A E AA AE R E A 
a Eu SU LECILEL ELI LET ULE ELL 8 5 m £ 
wre en fe t da AMARA RA RURAL AAA (Lg 9] R9 o£ 
viive 44 menga 2 wm EURER EBERLE 199978 ^e ott } 
Mone IBER LA E ARA RIA A Leo 
E ANNA NEUE NOAA EAS a DE 
"1*eese|]|]9o9784899 ** * ELBL Lg 4 LEE L gg gg 08g 8 ALLE EL ee eave 
hessen tt. PJAuLLLgBgusptdé4LL040095tim y 
'esoooorpraross.. PP ELIE REET ETE LE ES a 2 
error er fl or 900114 Ld => ""otoo ? 
we A/Z LI e«*»o/ «v f f Z4 [ 1 // mmm —h— 
"eco oet eos ovevo —— '"vsflgldllbilllleeef oL LT [vmm to t 
orten lio. oppl p os c* ^ 
arten 4 U EEE A AIN uP PT I [9 TC 
veros —om—os:e420090* 1f //Pg gg. ipa /pg ooo fA PP [9 t ct ee. D 
'eecovoocvoeoveoesetv i ZIAL iLi iit iLL 4 9 m. 9 G. 
doppoporsores.s IMA RARA IRA IA ARA RELLENA meter 
"essososcscesocot ILL LEA AR INP A RARA ALA con ee 
iS SR RELEASE RELE <q I 
A O ELE AAA Ne 3 
e Duma PILLE MARA RL LAI E AA “a4 4 
EEES AER TALA nv Me, 
EEE EEE Dette ae $ 
dee cia e CEs COL) ty mcs Sse eee oe Lee Te 
£440 0g 000007 (0040 1) 1] 1 g080 me ts (oc£424II7 Lg Lg 8g € 
PLEOF E89 LE C1 044406 490 [([4 matt onm ons ZI EI IT LEE $ 
Fil [(e- o" (2/06 P [l| rmi 442))lglllll)Ilf(—— 
Fee soococcelf (9Bllltn micum vy vever orte sd PL TPPFTL y 
Pilororncrnaa il dl o ev yy ot San meme sees oes reese el 
LEP Es Rete woe us SFLFLE LLG "rv, pror gro ts ARIS 4 
[ffavoorvrr nora sJ¿¿/79... swe 9$ 0:43.74 (49 ILCIF040 4 
Pietsovoovovecos lg lli itomas & ne ties Ln S 
F9 o oho ooa 7/7 f tto tm í]emg (79 t1 89 9 ¿PERROS g 
Pfecmecuocoses fofsssds Lee | "tim, es RRA RL 
Silgpewvoeeress fs sisi fA wevsweony svesuvasvususssssie ii & 
lloros 11111111440 COP Wosv a EEE EISEN 
££T44L E 0L 0240 9 4 EL01L 9 09 0404444 oon tm nm» m o LAA 
METTE EHE EAS EE E EH GEEEDD I Fu eism M Ep Roy 
HMLELTST0800 (90616 1L 0208 £200 C00444 400004 rro] m aan pll s 
Lgtg8010420LgL9 04000274424 0000074 04100410 0t 709 RT 3 
EEE oh Ne SE 
Ce ee A RAI REE ES SS eee wee yO 
Wd twee TH mat RIGA RL Lg Lg gg B gg 9 m e 
esiste UC ME FITTED DUST EE saecu T 
we wat cs, cm ví iib 222 PA 2/7 22 P9 P 79] £g gl o9 198 wm t 8 
vei sem ciewismse mies PEZPZPPEEPLE LLL LL B n mmo 
nrw ten at 4 "e DRRIEERE BER TER TG PE A 
Pa AAAI TOR M: AEE ACL TELET TINTE ra a ay 1 
-..— EEE EEE ED erw ww nu N 
AAA MA. RIBA AAA AMA AA ARA I II set nut y 
!w*ecoetoíeoc9ecceotw i4 PJLIljy//PggRIP/// 2g gg] / pg o. 
oo ort FSS ELFER SEER ELE ienne ve 79 G 
ty |]])oossos conte co RIMA RIMA MARIA AAA AA Ro à 
As EA ETERNA CETTE ELE EE Oe cet eae 
OO OD OO A ARABE RR OUNAE E ASS 
Pee eels S000) 81072026 velat tAE? Mnn G 
“roter ok runa LETTER ET ER EEE I EP aa 





FIGURE R-10 Backgrounds + White noise + Streaks 
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RESIDUAL SIGNAL AFTER WIENER FILTERING 
Figure R-10 


Noise + Streaks 
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FIGURE R-12 RESIDUAL SIGNAL AFTER ADAPTIVE RECURSIVE FILTERING 
White Noise + Streaks + Adaptation 





VIE SyNELUSTONS 


The study described herein has developed a new algorithm 
for the one-dimensional signal filtering problem and extends 
this to two-dimensional processing. It is an adaptive recur- 
sive filtering algorithm based on the steepest descent gradient 
method which employs the finite point square error for the 
gradient estimation rather than instantaneous square error. 

A simplified two-dimensional version of this algorithm 
is developed. It is designed to estimate the signal in real- 
time operation in cases where the statistics of both signal 
and corrupting noise are not available apriori. The algorithm 
learns the statistics and adapts even though it is not optimal, 
which means that it seeks the minimum of the error criterion. 

It should be noted that Widrow's nonrecursive adaptive filtering 
algorithm givesthe global minimum of performance criterion due 
to the fact that for the stationary process, the mean square 
error is the quadratic form of weight vectors, but for the re- 
cursive adaptive filter, local minima may be found instead of the 
global minimum. The computer simulation shows that for the examples 
considered here the algorithms presented learn the 
statistics of signal and adapt, Several points can be observed 
through the experimental results [see Figure R-l through R-12]. 
1) All the algorithms presented here give a satisfactory 

result after the transients die out even though they are 


not optimal, 


18055 





2) The algorithm which employs the finite point average 
square error for the gradient estimates gives more 
rapid convergence than Feintuck's algorithm. The 
possible reasons may be due to the fact that the output 
information is fed back and used for the filter coef- 
ficients updating process and the sensitivity information 
propagates through the recursive equation as the iteration 
proceeds, while Feintuck's algorithm discards the sensi- 
tivity information. 

E The algorithm developed here gives the best results among 
the various algorithms presented at the expense of 
complex hardware. Note that the required number of addi- 
tional sensitivity filters (equations (3-47), (3-48)) 
would be the number of filter coefficients, and due to the 
L point averaging process, additional storage elements are 
also needed. The possible reason for the good results 
may be due to the fact that the averaging process [equation 
(3-40)] for the gradient estimate gives a smaller 

error between true gradient and estimated 


gradients than the gradient estimate based on instantaneous 


square error does, while both give unbiased gradient 

estimates, 

Due to the emerging interest in adaptive recursive 
filters, further research on this subject may be worthwhile. 


The following are left open for further research. 
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1) 


2) 


3) 


4) 


5) 


Comparison of steady state performance of the recur- 
sive adaptive filter with the Kalman filtering tech- 
nique. It would lead to a better understanding of the 
performance of the recursive filter to express the 
cocer censo terms of steady-state Kalman 


co ans 


Mathematical derivation of the bound in step size of 
the filter coefficient updating process for convergence 
and stability. It is believed that this bound may be at- 


lcd seee Ue the constraints first such that the 


Es wor-sp zrormenee criterion decreases mono- 


tomically to a minimum as the iteration progresses. 


Modification of the algorıthms for the case that . partial 
statistics of signal or noise are available apriori. 
Derivation of the algorithm based on a different per- 
formance criterion such as maximum likelihood ratio, 
maximum signal to noise ratio, etc. 

Derivation of the algorithm based on the different mini- 
mization techniques such as Newton's method or Fletcher- 


Powell methods, etc., for a given performance criterion. 


07 





BSH OR ZREEERENGES 


E / Widrow, B., "Adaptive Filters," Aspects of Network and 


Ug! 


System Theory, R. E. Kalman and N. DeClaris, Eds., New 
York, Holt, Rinehart and Winston, 1970. 


ncn EE AeNdaptive Recursive LMS Filter," 
Proceedings of IEEE. November 1976. 


White, S. A., "An Adaptive Recursive Filter,"  Proceed- 
ings 9th Asilomar Conference on Circuits, Systems and 
Computers, November 19/5. 


Wrdrow, B., et al, "Adaptive Noise Cancelling: Principles 
and Applications," Proceedings of IEEE, December 1975. 


Blake, Ian F., and Thomas, John B., "The Linear Random 
Process,' Proceedings of IEEE, October 1968. 


Makhoul, John, "Linear Prediction: A Tutorial Review," 
Proceedings of IEEE, April 1975. 


Papoulis, A., "Probability, Random Variables and Sto- 
chastic Process," New York, McGraw-Hill, 1965. 


Weiner, N., "The Extrapolation, Interpolation and 
Smoothing of Stationary Time Series with Engineering 
Applications," John Wiley and Sons, Inc. New York, 
1949. 


Kalman, R. E., "A New Approach to Linear Filtering on 


eeresionerroplems,  ASME Trans., J. Basic Eng., 
oS March 1900, pp. 34-35. 


108 





D. 


INITIAL DISTRIBUTION LIST 
No. Copies 


Defense Documentation Center 2 
Cameron Station 
Alexandria, Virginia 22314 


Library, Code 0142 2 
Naval Postgraduate School 
Monterey, California 93940 


Department Chairman, Code 62 D 
Department of Electrical Engineering 

Naval Postgraduate School 

Monterey, California 93940 


Professor Sydney R. Parker, Code 62Px(Thesis Advisor) 1 
Department of Electrical Engineering 

Naval Postgraduate School 

Monterey, California 95940 


Lieutenant Soon-Ju Ko, ROKN 3 
ADO 2 DONG 
CHEYO  CHEIUDO 590 KOREA 


Brotes some Jury 1 
Iepartweuntsoi Electrical Engineering and 

Computer Science 

University of California 

Berkeley, CA 94720 


Erorfessor=sapnjit K. Mitra 1 
Department of Electrical Engineering 

and Computer Science 

Deuezersıty of California 

santa Barbara, CA 93106 


Division of Personal Education and Training 2 
Republic of Korea Naval Headquarters 

DAEBANG-DONG YOUNGDEUNGPO-KU 

SEOUL 715-18, KOREA 


Baruch, Even-or je 
SMC 2190 

Naval Postgraduate School 

Monterey, CA 93940 


Pereira, Jose 1 
SMC 2610 

Naval Postgraduate School 

Monterey, CA 93940 


109 








Ko |. 
An adaptive recur- 
sive filter. 





thesK7 1625 
adaptive recursive filter. 


DUDLEY KNOX LIBRARY 





